[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:03:05 dist-test-slave-t3q3 krb5kdc[1535](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:03:05 dist-test-slave-t3q3 krb5kdc[1535](info): set up 2 sockets
Aug 08 13:03:05 dist-test-slave-t3q3 krb5kdc[1535](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 13:03:07.213476 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.451s user 0.008s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:03:07 dist-test-slave-t3q3 krb5kdc[1535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:03:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:03:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:03:07.417150 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:42275
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:41237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:42275
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:07.911051 1559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:07.911455 1559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:07.911551 1559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:07.919852 1559 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:03:07.919970 1559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:07.920019 1559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:07.920132 1559 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:03:07.920198 1559 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:03:07.931221 1559 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:41237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:42275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:42275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:07.933202 1559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:07.935163 1559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:07.955770 1568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:07.956341 1572 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:07.957683 1566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:07.960927 1559 server_base.cc:1034] running on GCE node
I20260808 13:03:07.962085 1559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:07.964534 1559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:07.965792 1559 hybrid_clock.cc:648] HybridClock initialized: now 1786194187965694 us; error 123 us; skew 500 ppm
I20260808 13:03:07.966262 1559 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:07 dist-test-slave-t3q3 krb5kdc[1535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194187, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:07.971025 1559 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:03:07.973623 1559 webserver.cc:460] Webserver started at http://127.1.124.190:36231/ using document root <none> and password file <none>
I20260808 13:03:07.974685 1559 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:07.974846 1559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:07.975239 1559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:07.979652 1559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "0db469ac00234f5cb2c265bd717615cb"
format_stamp: "Formatted at 2026-08-08 13:03:07 on dist-test-slave-t3q3"
server_key: "f1db4f0602531b470fc72bfb9f780502"
server_key_iv: "d6c86e54a0f6d726f2b810f2fcdc1e74"
server_key_version: "encryptionkey@0"
I20260808 13:03:07.980806 1559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "0db469ac00234f5cb2c265bd717615cb"
format_stamp: "Formatted at 2026-08-08 13:03:07 on dist-test-slave-t3q3"
server_key: "f1db4f0602531b470fc72bfb9f780502"
server_key_iv: "d6c86e54a0f6d726f2b810f2fcdc1e74"
server_key_version: "encryptionkey@0"
I20260808 13:03:07.987528 1559 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260808 13:03:07.991866 1581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:07.993619 1559 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 13:03:07.993854 1559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "0db469ac00234f5cb2c265bd717615cb"
format_stamp: "Formatted at 2026-08-08 13:03:07 on dist-test-slave-t3q3"
server_key: "f1db4f0602531b470fc72bfb9f780502"
server_key_iv: "d6c86e54a0f6d726f2b810f2fcdc1e74"
server_key_version: "encryptionkey@0"
I20260808 13:03:07.994153 1559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:08.027024 1559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:08.032342 1559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:08.032712 1559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:08.066493 1559 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:42275
I20260808 13:03:08.066493 1656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:42275 every 8 connection(s)
I20260808 13:03:08.068512 1559 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:03:08.074205 1657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:08.074458 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 1559
I20260808 13:03:08.074981 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:03:08.078282 1522 external_mini_cluster.cc:1428] Setting key dbf1652c2879316d25ed01d1b5522f28
I20260808 13:03:08.091365 1657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb: Bootstrap starting.
I20260808 13:03:08.094924 1657 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:08.096138 1657 log.cc:824] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:08.100181 1657 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb: No bootstrap required, opened a new log
I20260808 13:03:08.109052 1657 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0db469ac00234f5cb2c265bd717615cb" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42275 } }
I20260808 13:03:08.109549 1657 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:08.109709 1657 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0db469ac00234f5cb2c265bd717615cb, State: Initialized, Role: FOLLOWER
I20260808 13:03:08.110532 1657 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0db469ac00234f5cb2c265bd717615cb" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42275 } }
I20260808 13:03:08.110901 1657 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:03:08.111155 1657 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:03:08.111410 1657 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb [term 0 FOLLOWER]: Advancing to term 1
Aug 08 13:03:08 dist-test-slave-t3q3 krb5kdc[1535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:08.125056 1663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:08.084768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37150 (local address 127.1.124.190:42275)
0808 13:03:08.087562 (+ 2794us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:08.087586 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:08.087641 (+ 55us) server_negotiation.cc:407] Connection header received
0808 13:03:08.098307 (+ 10666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:08.098331 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:08.098942 (+ 611us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:08.099346 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:08.102769 (+ 3423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:08.104676 (+ 1907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:08.106758 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:08.107658 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:08.107980 (+ 322us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:08.115279 (+ 7299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:08.115287 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:08.115302 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:08.115344 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:08.118868 (+ 3524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:08.119838 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:08.119845 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:08.119850 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:08.120016 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:08.120678 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:08.120686 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:08.120690 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:08.121394 (+ 704us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:08.121856 (+ 462us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:08.122770 (+ 914us) server_negotiation.cc:299] Negotiation successful
0808 13:03:08.123277 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2489,"thread_start_us":2081,"threads_started":1}
I20260808 13:03:08.149519 1657 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0db469ac00234f5cb2c265bd717615cb" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42275 } }
I20260808 13:03:08.150424 1657 leader_election.cc:302] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0db469ac00234f5cb2c265bd717615cb; no voters:
I20260808 13:03:08.151837 1657 leader_election.cc:288] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:03:08.151911 1665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:08.154198 1665 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb [term 1 LEADER]: Becoming Leader. State: Replica: 0db469ac00234f5cb2c265bd717615cb, State: Running, Role: LEADER
I20260808 13:03:08.154896 1665 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0db469ac00234f5cb2c265bd717615cb" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42275 } }
I20260808 13:03:08.156159 1657 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:03:08.159683 1672 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0db469ac00234f5cb2c265bd717615cb. Latest consensus state: current_term: 1 leader_uuid: "0db469ac00234f5cb2c265bd717615cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0db469ac00234f5cb2c265bd717615cb" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42275 } } }
I20260808 13:03:08.160435 1672 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:08.159763 1671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0db469ac00234f5cb2c265bd717615cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0db469ac00234f5cb2c265bd717615cb" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42275 } } }
I20260808 13:03:08.160921 1671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:08.165345 1678 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:03:08.172134 1678 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:03:08.183394 1678 catalog_manager.cc:1347] Generated new cluster ID: 55a490e49ef34cbfa1a8dd167c4bdfba
I20260808 13:03:08.183627 1678 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:03:08.211297 1678 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:03:08.212899 1678 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:03:08.222370 1678 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0db469ac00234f5cb2c265bd717615cb: Generated new TSK 0
I20260808 13:03:08.223466 1678 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.124.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
W20260808 13:03:08.812886 1522 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.1.124.129: real 0.574s user 0.001s sys 0.017s
I20260808 13:03:08.814076 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42275
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:41237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:09.214951 1695 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:09.215346 1695 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:09.215549 1695 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:09.224663 1695 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:09.224838 1695 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:09.225042 1695 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:03:09.236688 1695 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:41237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42275
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:09.238257 1695 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:09.239759 1695 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:09.255673 1706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:09.255625 1702 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:09.255725 1704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:09.256614 1695 server_base.cc:1034] running on GCE node
I20260808 13:03:09.257372 1695 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:09.258327 1695 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:09.259573 1695 hybrid_clock.cc:648] HybridClock initialized: now 1786194189259502 us; error 57 us; skew 500 ppm
I20260808 13:03:09.259927 1695 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:09 dist-test-slave-t3q3 krb5kdc[1535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194189, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:09.263262 1695 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:03:09.264911 1695 webserver.cc:460] Webserver started at http://127.1.124.129:33731/ using document root <none> and password file <none>
I20260808 13:03:09.265748 1695 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:09.265861 1695 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:09.266175 1695 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:09.269409 1695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "37cdf212334a4343952ec047dd7ae4f2"
format_stamp: "Formatted at 2026-08-08 13:03:09 on dist-test-slave-t3q3"
server_key: "aa7b6ae7c94545b01edfae2624194d4f"
server_key_iv: "ac1bf9500378c2c374d8319c86186f07"
server_key_version: "encryptionkey@0"
I20260808 13:03:09.270160 1695 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "37cdf212334a4343952ec047dd7ae4f2"
format_stamp: "Formatted at 2026-08-08 13:03:09 on dist-test-slave-t3q3"
server_key: "aa7b6ae7c94545b01edfae2624194d4f"
server_key_iv: "ac1bf9500378c2c374d8319c86186f07"
server_key_version: "encryptionkey@0"
I20260808 13:03:09.275875 1695 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 13:03:09.279861 1719 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:09.281450 1695 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:03:09.281653 1695 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "37cdf212334a4343952ec047dd7ae4f2"
format_stamp: "Formatted at 2026-08-08 13:03:09 on dist-test-slave-t3q3"
server_key: "aa7b6ae7c94545b01edfae2624194d4f"
server_key_iv: "ac1bf9500378c2c374d8319c86186f07"
server_key_version: "encryptionkey@0"
I20260808 13:03:09.281879 1695 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:09.302790 1695 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:09.307731 1695 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:09.308035 1695 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:09.309158 1695 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:03:09.310560 1695 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:09.310636 1695 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:09.310808 1695 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:09.310870 1695 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:09.351622 1695 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:36955
I20260808 13:03:09.351675 1883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:36955 every 8 connection(s)
I20260808 13:03:09.353008 1695 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:03:09.357395 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 1695
I20260808 13:03:09.357682 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:03:09.358688 1522 external_mini_cluster.cc:1428] Setting key 805140cde36f6f9a34f5840c0e336765
Aug 08 13:03:09 dist-test-slave-t3q3 krb5kdc[1535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194189, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:09.378727 1886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:09.355839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:44197 (local address 127.1.124.190:42275)
0808 13:03:09.356331 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:09.356335 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:09.357589 (+ 1254us) server_negotiation.cc:407] Connection header received
0808 13:03:09.360324 (+ 2735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:09.360329 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:09.360395 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:09.360492 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:09.362553 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:09.363330 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:09.366329 (+ 2999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:09.366796 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:09.366929 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:09.372003 (+ 5074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:09.372008 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:09.372011 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:09.372043 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:09.374689 (+ 2646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:09.375724 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:09.375729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:09.375731 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:09.375858 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:09.376375 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:09.376379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:09.376381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:09.376796 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:09.376958 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:09.377913 (+ 955us) server_negotiation.cc:299] Negotiation successful
0808 13:03:09.378173 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":284,"threads_started":1}
I20260808 13:03:09.379323 1887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:09.356332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:42275 (local address 127.1.124.129:44197)
0808 13:03:09.357308 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:09.357376 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:09.360010 (+ 2634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:09.360782 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:09.360790 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:09.361511 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:09.362339 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:09.362369 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:09.363504 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:09.363511 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:09.366173 (+ 2662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:09.366191 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:09.367067 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:09.367074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:09.367449 (+ 375us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:09.368437 (+ 988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:09.368486 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:09.371767 (+ 3281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:09.374839 (+ 3072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:09.374851 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:09.374871 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:09.375594 (+ 723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:09.375967 (+ 373us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:09.375975 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:09.375982 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:09.376258 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:09.376954 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:09.376965 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:09.377444 (+ 479us) client_negotiation.cc:769] Sending connection context
0808 13:03:09.377786 (+ 342us) client_negotiation.cc:240] Negotiation successful
0808 13:03:09.378230 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":272,"threads_started":1}
I20260808 13:03:09.381034 1884 heartbeater.cc:347] Connected to a master server at 127.1.124.190:42275
I20260808 13:03:09.381516 1884 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.1.124.130@KRBTEST.COM; defaulting to no policy
I20260808 13:03:09.382586 1884 heartbeater.cc:511] Master 127.1.124.190:42275 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.130@KRBTEST.COM" created.
I20260808 13:03:09.386554 1601 ts_manager.cc:194] Registered new tserver with Master: 37cdf212334a4343952ec047dd7ae4f2 (127.1.124.129:36955)
I20260808 13:03:09.388854 1601 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:44197
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
I20260808 13:03:09.410909 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42275
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:41237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:09.782121 1890 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:09.782397 1890 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:09.782495 1890 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:09.790266 1890 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:09.790421 1890 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:09.790591 1890 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:03:09.800880 1890 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:41237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42275
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:09.802312 1890 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:09.803746 1890 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:09.818092 1898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:09.818149 1904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:09.818209 1901 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:09.819312 1890 server_base.cc:1034] running on GCE node
I20260808 13:03:09.820005 1890 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:09.821053 1890 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:09.822283 1890 hybrid_clock.cc:648] HybridClock initialized: now 1786194189822227 us; error 48 us; skew 500 ppm
I20260808 13:03:09.822616 1890 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:09 dist-test-slave-t3q3 krb5kdc[1535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194189, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:09.826000 1890 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:03:09.827667 1890 webserver.cc:460] Webserver started at http://127.1.124.130:39111/ using document root <none> and password file <none>
I20260808 13:03:09.828439 1890 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:09.828583 1890 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:09.828902 1890 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:09.831986 1890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "561cf8c7b2b14b53b96c0f9ba506472a"
format_stamp: "Formatted at 2026-08-08 13:03:09 on dist-test-slave-t3q3"
server_key: "4617708a23d9b97ca404c0440be21d6f"
server_key_iv: "ab35f6d56f85b0da1630650136dcb050"
server_key_version: "encryptionkey@0"
I20260808 13:03:09.832808 1890 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "561cf8c7b2b14b53b96c0f9ba506472a"
format_stamp: "Formatted at 2026-08-08 13:03:09 on dist-test-slave-t3q3"
server_key: "4617708a23d9b97ca404c0440be21d6f"
server_key_iv: "ab35f6d56f85b0da1630650136dcb050"
server_key_version: "encryptionkey@0"
I20260808 13:03:09.838346 1890 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:03:09.841811 1913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:09.843115 1890 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:03:09.843293 1890 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "561cf8c7b2b14b53b96c0f9ba506472a"
format_stamp: "Formatted at 2026-08-08 13:03:09 on dist-test-slave-t3q3"
server_key: "4617708a23d9b97ca404c0440be21d6f"
server_key_iv: "ab35f6d56f85b0da1630650136dcb050"
server_key_version: "encryptionkey@0"
I20260808 13:03:09.843493 1890 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:09.875449 1890 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:09.879894 1890 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:09.880307 1890 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:09.881357 1890 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:03:09.882959 1890 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:09.883056 1890 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:09.883229 1890 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:09.883311 1890 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:09.922842 1890 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:34815
I20260808 13:03:09.922814 2076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:34815 every 8 connection(s)
I20260808 13:03:09.924278 1890 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:03:09.933235 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 1890
I20260808 13:03:09.933437 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:03:09.933934 1522 external_mini_cluster.cc:1428] Setting key 6c3d5aa009f393568e2eea6e21c83745
Aug 08 13:03:09 dist-test-slave-t3q3 krb5kdc[1535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194189, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:09.947804 2080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:09.927190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:39277 (local address 127.1.124.190:42275)
0808 13:03:09.927663 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:09.927667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:09.928653 (+ 986us) server_negotiation.cc:407] Connection header received
0808 13:03:09.930572 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:09.930579 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:09.930671 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:09.930791 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:09.932772 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:09.933663 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:09.935467 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:09.936054 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:09.936237 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:09.940656 (+ 4419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:09.940662 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:09.940665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:09.940697 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:09.943791 (+ 3094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:09.944618 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:09.944627 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:09.944633 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:09.944912 (+ 279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:09.945417 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:09.945425 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:09.945431 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:09.945963 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:09.946130 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:09.947020 (+ 890us) server_negotiation.cc:299] Negotiation successful
0808 13:03:09.947291 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":261,"threads_started":1}
I20260808 13:03:09.948884 2081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:09.927703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:42275 (local address 127.1.124.130:39277)
0808 13:03:09.928465 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:09.928505 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:09.930303 (+ 1798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:09.931020 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:09.931029 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:09.931694 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:09.932609 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:09.932628 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:09.933812 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:09.933817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:09.935314 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:09.935332 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:09.936329 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:09.936338 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:09.936579 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:09.937352 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:09.937374 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:09.940395 (+ 3021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:09.943951 (+ 3556us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:09.943958 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:09.943972 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:09.944476 (+ 504us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:09.945068 (+ 592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:09.945072 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:09.945075 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:09.945288 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:09.946123 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:09.946130 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:09.946514 (+ 384us) client_negotiation.cc:769] Sending connection context
0808 13:03:09.946851 (+ 337us) client_negotiation.cc:240] Negotiation successful
0808 13:03:09.947390 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":354,"threads_started":1}
I20260808 13:03:09.950685 2078 heartbeater.cc:347] Connected to a master server at 127.1.124.190:42275
I20260808 13:03:09.951040 2078 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:09.951916 2078 heartbeater.cc:511] Master 127.1.124.190:42275 requested a full tablet report, sending...
I20260808 13:03:09.953575 1601 ts_manager.cc:194] Registered new tserver with Master: 561cf8c7b2b14b53b96c0f9ba506472a (127.1.124.130:34815)
I20260808 13:03:09.954591 1601 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:39277
WARNING: no policy specified for kudu/127.1.124.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
I20260808 13:03:09.982764 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42275
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:41237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:10.339030 2086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:10.339308 2086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:10.339473 2086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:10.346916 2086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:10.347024 2086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:10.347195 2086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:03:10.358105 2086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:41237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42275
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:10.359273 2086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:10.360940 2086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:10.375090 2091 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:10.375152 2095 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:10.375205 2093 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:10.376325 2086 server_base.cc:1034] running on GCE node
I20260808 13:03:10.376942 2086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:10.377908 2086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:10.379148 2086 hybrid_clock.cc:648] HybridClock initialized: now 1786194190379073 us; error 53 us; skew 500 ppm
I20260808 13:03:10.379504 2086 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:10 dist-test-slave-t3q3 krb5kdc[1535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194190, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:10.387503 2086 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:03:10.389143 2086 webserver.cc:460] Webserver started at http://127.1.124.131:45273/ using document root <none> and password file <none>
I20260808 13:03:10.389894 2086 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:10.390033 2086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:10.390362 2086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:10.392264 1884 heartbeater.cc:503] Master 127.1.124.190:42275 was elected leader, sending a full tablet report...
I20260808 13:03:10.393638 2086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "26369825d0794f898fe540592a97f157"
format_stamp: "Formatted at 2026-08-08 13:03:10 on dist-test-slave-t3q3"
server_key: "223b98288cbe58b332d7f620c25c50e5"
server_key_iv: "9845dac21266431570dc71cc14720301"
server_key_version: "encryptionkey@0"
I20260808 13:03:10.394637 2086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "26369825d0794f898fe540592a97f157"
format_stamp: "Formatted at 2026-08-08 13:03:10 on dist-test-slave-t3q3"
server_key: "223b98288cbe58b332d7f620c25c50e5"
server_key_iv: "9845dac21266431570dc71cc14720301"
server_key_version: "encryptionkey@0"
I20260808 13:03:10.400504 2086 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 13:03:10.403997 2107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:10.405542 2086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:03:10.405719 2086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "26369825d0794f898fe540592a97f157"
format_stamp: "Formatted at 2026-08-08 13:03:10 on dist-test-slave-t3q3"
server_key: "223b98288cbe58b332d7f620c25c50e5"
server_key_iv: "9845dac21266431570dc71cc14720301"
server_key_version: "encryptionkey@0"
I20260808 13:03:10.405925 2086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:10.425211 2086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:10.429404 2086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:10.429710 2086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:10.430713 2086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:03:10.432149 2086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:10.432250 2086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:10.432396 2086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:10.432466 2086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:10.471609 2086 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:42167
I20260808 13:03:10.471675 2273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:42167 every 8 connection(s)
I20260808 13:03:10.473026 2086 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:03:10.476310 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 2086
I20260808 13:03:10.476500 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:03:10.477154 1522 external_mini_cluster.cc:1428] Setting key 0811b202a694729918fddc0ae8767acf
Aug 08 13:03:10 dist-test-slave-t3q3 krb5kdc[1535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194190, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:10.495038 2281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:10.475799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:60795 (local address 127.1.124.190:42275)
0808 13:03:10.476375 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:10.476382 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:10.477545 (+ 1163us) server_negotiation.cc:407] Connection header received
0808 13:03:10.480093 (+ 2548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:10.480102 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:10.480200 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:10.480293 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:10.482101 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:10.482882 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:10.484559 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:10.484993 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:10.485125 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:10.488893 (+ 3768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:10.488898 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:10.488902 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:10.488936 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:10.491528 (+ 2592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:10.492334 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:10.492338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:10.492340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:10.492457 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:10.492940 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:10.492944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:10.492946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:10.493370 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:10.493529 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:10.494472 (+ 943us) server_negotiation.cc:299] Negotiation successful
0808 13:03:10.494734 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":295,"threads_started":1}
I20260808 13:03:10.496161 2283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:10.476393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:42275 (local address 127.1.124.131:60795)
0808 13:03:10.477317 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:10.477404 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:10.479721 (+ 2317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:10.480547 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:10.480555 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:10.481089 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:10.481787 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:10.481814 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:10.483041 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:10.483044 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:10.484416 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:10.484429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:10.485217 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:10.485222 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:10.485421 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:10.486118 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:10.486147 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:10.488642 (+ 2495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:10.491673 (+ 3031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:10.491680 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:10.491695 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:10.492217 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:10.492565 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:10.492570 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:10.492574 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:10.492802 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:10.493524 (+ 722us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:10.493531 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:10.494000 (+ 469us) client_negotiation.cc:769] Sending connection context
0808 13:03:10.494327 (+ 327us) client_negotiation.cc:240] Negotiation successful
0808 13:03:10.494896 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":331,"threads_started":1}
I20260808 13:03:10.497627 2274 heartbeater.cc:347] Connected to a master server at 127.1.124.190:42275
I20260808 13:03:10.497959 2274 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:10.499046 2274 heartbeater.cc:511] Master 127.1.124.190:42275 requested a full tablet report, sending...
I20260808 13:03:10.500593 1601 ts_manager.cc:194] Registered new tserver with Master: 26369825d0794f898fe540592a97f157 (127.1.124.131:42167)
I20260808 13:03:10.501420 1601 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:60795
I20260808 13:03:10.506965 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 13:03:10 dist-test-slave-t3q3 krb5kdc[1535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194190, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 13:03:10 dist-test-slave-t3q3 krb5kdc[1535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194190, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.124.129@KRBTEST.COM
I20260808 13:03:10.558749 2291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:10.542574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45974 (local address 127.1.124.129:36955)
0808 13:03:10.543385 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:10.543394 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:10.543428 (+ 34us) server_negotiation.cc:407] Connection header received
0808 13:03:10.543510 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:10.543517 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:10.543734 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:10.543906 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:10.545701 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:10.546753 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:10.548182 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:10.548754 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:10.548897 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:10.553064 (+ 4167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:10.553103 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:10.553118 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:10.553149 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:10.555923 (+ 2774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:10.556607 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:10.556614 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:10.556616 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:10.556742 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:10.557140 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:10.557144 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:10.557146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:10.557511 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:10.557828 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:10.558141 (+ 313us) server_negotiation.cc:299] Negotiation successful
0808 13:03:10.558405 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":400,"threads_started":1}
W20260808 13:03:10.560190 1764 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45974
Aug 08 13:03:10 dist-test-slave-t3q3 krb5kdc[1535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 13:03:10 dist-test-slave-t3q3 krb5kdc[1535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.129@KRBTEST.COM
I20260808 13:03:10.611107 2291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:10.595808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45978 (local address 127.1.124.129:36955)
0808 13:03:10.596118 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:10.596130 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:10.596438 (+ 308us) server_negotiation.cc:407] Connection header received
0808 13:03:10.596666 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:10.596676 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:10.596786 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:10.596906 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:10.598485 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:10.599352 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:10.600739 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:10.601173 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:10.601331 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:10.605447 (+ 4116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:10.605452 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:10.605456 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:10.605488 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:10.608011 (+ 2523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:10.608836 (+ 825us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:10.608840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:10.608843 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:10.609033 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:10.609478 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:10.609482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:10.609484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:10.609836 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:10.610076 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:10.610412 (+ 336us) server_negotiation.cc:299] Negotiation successful
0808 13:03:10.610661 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260808 13:03:10.613325 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 1695
I20260808 13:03:10.627826 2291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:10.614729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45980 (local address 127.1.124.129:36955)
0808 13:03:10.614901 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:10.614906 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:10.615230 (+ 324us) server_negotiation.cc:407] Connection header received
0808 13:03:10.615363 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:10.615367 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:10.615427 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:10.615526 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:10.616925 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:10.617673 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:10.619178 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:10.619591 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:10.619691 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:10.622171 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:10.622176 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:10.622180 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:10.622235 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:10.625176 (+ 2941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:10.625841 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:10.625847 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:10.625851 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:10.625990 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:10.626379 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:10.626387 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:10.626390 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:10.626666 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:10.626924 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:10.627238 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 13:03:10.627453 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 13:03:10.628750 1878 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:10.720188 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 1695
I20260808 13:03:10.751083 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 1890
Aug 08 13:03:10 dist-test-slave-t3q3 krb5kdc[1535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.130@KRBTEST.COM
I20260808 13:03:10.768399 2300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:10.752187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52638 (local address 127.1.124.130:34815)
0808 13:03:10.752722 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:10.752729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:10.752754 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:03:10.752815 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:10.752821 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:10.753041 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:10.753231 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:10.754463 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:10.755502 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:10.757225 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:10.757921 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:10.758094 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:10.762460 (+ 4366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:10.762473 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:10.762487 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:10.762520 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:10.765556 (+ 3036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:10.766286 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:10.766293 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:10.766296 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:10.766488 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:10.766900 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:10.766919 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:10.766922 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:10.767291 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:10.767566 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:10.767800 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 13:03:10.768010 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":264,"threads_started":1}
I20260808 13:03:10.769851 2071 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:10.853600 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 1890
I20260808 13:03:10.879629 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 2086
Aug 08 13:03:10 dist-test-slave-t3q3 krb5kdc[1535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.131@KRBTEST.COM
I20260808 13:03:10.900419 2304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:10.880883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33754 (local address 127.1.124.131:42167)
0808 13:03:10.881582 (+ 699us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:10.881590 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:10.881619 (+ 29us) server_negotiation.cc:407] Connection header received
0808 13:03:10.881698 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:10.881704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:10.881909 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:10.882052 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:10.883358 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:10.884541 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:10.886666 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:10.887758 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:10.887929 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:10.894167 (+ 6238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:10.894179 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:10.894194 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:10.894233 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:10.897458 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:10.898099 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:10.898105 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:10.898108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:10.898235 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:10.898711 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:10.898715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:10.898718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:10.899082 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:10.899328 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:10.899706 (+ 378us) server_negotiation.cc:299] Negotiation successful
0808 13:03:10.899946 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":385,"threads_started":1}
I20260808 13:03:10.901739 2268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:10.987619 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 2086
I20260808 13:03:11.010026 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 1559
I20260808 13:03:11.011349 1650 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:11.117530 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 1559
2026-08-08T13:03:11Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (5424 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:03:11 dist-test-slave-t3q3 krb5kdc[2312](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:03:11 dist-test-slave-t3q3 krb5kdc[2312](info): set up 2 sockets
Aug 08 13:03:11 dist-test-slave-t3q3 krb5kdc[2312](info): commencing operation
krb5kdc: starting...
W20260808 13:03:12.427898 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.258s user 0.000s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:03:12 dist-test-slave-t3q3 krb5kdc[2312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:03:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:03:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:03:12.633400 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:42789
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:42951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:42789
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:12.966774 2329 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:12.967168 2329 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:12.967329 2329 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:12.974856 2329 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:03:12.975044 2329 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:12.975168 2329 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:12.975276 2329 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:03:12.975348 2329 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:03:12.985361 2329 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:42951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:42789
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:42789
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:12.986781 2329 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:12.988170 2329 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:13.000931 2336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:13.000974 2337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:13.001015 2342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:13.002229 2329 server_base.cc:1034] running on GCE node
I20260808 13:03:13.002967 2329 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:13.004424 2329 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:13.005653 2329 hybrid_clock.cc:648] HybridClock initialized: now 1786194193005604 us; error 32 us; skew 500 ppm
I20260808 13:03:13.006032 2329 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:13 dist-test-slave-t3q3 krb5kdc[2312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194193, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:13.009337 2329 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:03:13.010799 2329 webserver.cc:460] Webserver started at http://127.1.124.190:39151/ using document root <none> and password file <none>
I20260808 13:03:13.011466 2329 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:13.011569 2329 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:13.011868 2329 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:13.015062 2329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "fc083444bd9f4f2486d71c3b30a28177"
format_stamp: "Formatted at 2026-08-08 13:03:13 on dist-test-slave-t3q3"
server_key: "77525112270db4771bd2bf97259627f5"
server_key_iv: "426f5b08a2862cf9d2c2b19dd0db44f8"
server_key_version: "encryptionkey@0"
I20260808 13:03:13.015810 2329 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "fc083444bd9f4f2486d71c3b30a28177"
format_stamp: "Formatted at 2026-08-08 13:03:13 on dist-test-slave-t3q3"
server_key: "77525112270db4771bd2bf97259627f5"
server_key_iv: "426f5b08a2862cf9d2c2b19dd0db44f8"
server_key_version: "encryptionkey@0"
I20260808 13:03:13.021067 2329 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260808 13:03:13.024748 2351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:13.026213 2329 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:03:13.026422 2329 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "fc083444bd9f4f2486d71c3b30a28177"
format_stamp: "Formatted at 2026-08-08 13:03:13 on dist-test-slave-t3q3"
server_key: "77525112270db4771bd2bf97259627f5"
server_key_iv: "426f5b08a2862cf9d2c2b19dd0db44f8"
server_key_version: "encryptionkey@0"
I20260808 13:03:13.026636 2329 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:13.048383 2329 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:13.052546 2329 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:13.052857 2329 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:13.071708 2329 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:42789
I20260808 13:03:13.071741 2433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:42789 every 8 connection(s)
I20260808 13:03:13.073372 2329 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:03:13.078341 2434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:13.083679 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 2329
I20260808 13:03:13.083892 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:03:13.084707 1522 external_mini_cluster.cc:1428] Setting key 5d787b380d279e5d31f895bd0fbc0ddf
I20260808 13:03:13.089954 2434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177: Bootstrap starting.
I20260808 13:03:13.093835 2434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:13.094877 2434 log.cc:824] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:13.098037 2434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177: No bootstrap required, opened a new log
Aug 08 13:03:13 dist-test-slave-t3q3 krb5kdc[2312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:13.106141 2434 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc083444bd9f4f2486d71c3b30a28177" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42789 } }
I20260808 13:03:13.106550 2434 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:13.106686 2434 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc083444bd9f4f2486d71c3b30a28177, State: Initialized, Role: FOLLOWER
I20260808 13:03:13.107388 2434 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc083444bd9f4f2486d71c3b30a28177" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42789 } }
I20260808 13:03:13.107676 2434 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:03:13.107833 2434 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:03:13.108037 2434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:13.108775 2439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:13.086953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34040 (local address 127.1.124.190:42789)
0808 13:03:13.087861 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:13.087873 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:13.087922 (+ 49us) server_negotiation.cc:407] Connection header received
0808 13:03:13.089722 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:13.089746 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:13.090204 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:13.090547 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:13.092536 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:13.093881 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:13.095259 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:13.096484 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:13.096690 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:13.100823 (+ 4133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:13.100831 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:13.100849 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:13.100888 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:13.104117 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:13.104823 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:13.104830 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:13.104840 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:13.105017 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:13.105413 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:13.105419 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:13.105424 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:13.105971 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:13.106285 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:13.106818 (+ 533us) server_negotiation.cc:299] Negotiation successful
0808 13:03:13.107403 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":441,"threads_started":1}
I20260808 13:03:13.111047 2434 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc083444bd9f4f2486d71c3b30a28177" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42789 } }
I20260808 13:03:13.111624 2434 leader_election.cc:302] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc083444bd9f4f2486d71c3b30a28177; no voters:
I20260808 13:03:13.112329 2434 leader_election.cc:288] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:03:13.112452 2441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:13.113216 2441 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177 [term 1 LEADER]: Becoming Leader. State: Replica: fc083444bd9f4f2486d71c3b30a28177, State: Running, Role: LEADER
I20260808 13:03:13.113761 2441 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc083444bd9f4f2486d71c3b30a28177" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42789 } }
I20260808 13:03:13.114476 2434 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:03:13.116540 2443 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc083444bd9f4f2486d71c3b30a28177" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc083444bd9f4f2486d71c3b30a28177" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42789 } } }
I20260808 13:03:13.116925 2443 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177 [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:13.117480 2447 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc083444bd9f4f2486d71c3b30a28177. Latest consensus state: current_term: 1 leader_uuid: "fc083444bd9f4f2486d71c3b30a28177" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc083444bd9f4f2486d71c3b30a28177" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42789 } } }
I20260808 13:03:13.117800 2447 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177 [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:13.117959 2450 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:03:13.124818 2450 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:03:13.134486 2450 catalog_manager.cc:1347] Generated new cluster ID: 0cf4686e26ca45d399140c1d0f6dc2a6
I20260808 13:03:13.134631 2450 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:03:13.163470 2450 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:03:13.164948 2450 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:03:13.172147 2450 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc083444bd9f4f2486d71c3b30a28177: Generated new TSK 0
I20260808 13:03:13.173107 2450 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.124.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
I20260808 13:03:13.234266 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42789
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:42951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:13.566781 2474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:13.567134 2474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:13.567325 2474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:13.574553 2474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:13.574712 2474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:13.574911 2474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:03:13.584673 2474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:42951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42789
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:13.586004 2474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:13.587380 2474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:13.600862 2484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:13.600862 2482 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:13.600991 2486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:13.601898 2474 server_base.cc:1034] running on GCE node
I20260808 13:03:13.602655 2474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:13.603610 2474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:13.604872 2474 hybrid_clock.cc:648] HybridClock initialized: now 1786194193604819 us; error 58 us; skew 500 ppm
I20260808 13:03:13.605294 2474 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:13 dist-test-slave-t3q3 krb5kdc[2312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194193, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:13.609227 2474 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:03:13.610929 2474 webserver.cc:460] Webserver started at http://127.1.124.129:38587/ using document root <none> and password file <none>
I20260808 13:03:13.611718 2474 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:13.611892 2474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:13.612324 2474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:13.615790 2474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "b2c37af441a34b67abd48bc870895267"
format_stamp: "Formatted at 2026-08-08 13:03:13 on dist-test-slave-t3q3"
server_key: "402da70f3b097987b201b28a8e16c383"
server_key_iv: "83e79e96f36c9c936fba1f042b6e11ff"
server_key_version: "encryptionkey@0"
I20260808 13:03:13.616761 2474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "b2c37af441a34b67abd48bc870895267"
format_stamp: "Formatted at 2026-08-08 13:03:13 on dist-test-slave-t3q3"
server_key: "402da70f3b097987b201b28a8e16c383"
server_key_iv: "83e79e96f36c9c936fba1f042b6e11ff"
server_key_version: "encryptionkey@0"
I20260808 13:03:13.622767 2474 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260808 13:03:13.626753 2498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:13.628396 2474 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:03:13.628633 2474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "b2c37af441a34b67abd48bc870895267"
format_stamp: "Formatted at 2026-08-08 13:03:13 on dist-test-slave-t3q3"
server_key: "402da70f3b097987b201b28a8e16c383"
server_key_iv: "83e79e96f36c9c936fba1f042b6e11ff"
server_key_version: "encryptionkey@0"
I20260808 13:03:13.628826 2474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:13.650308 2474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:13.654595 2474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:13.654912 2474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:13.655959 2474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:03:13.657361 2474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:13.657441 2474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:13.657622 2474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:13.657685 2474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:13.693742 2474 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:35363
I20260808 13:03:13.693778 2661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:35363 every 8 connection(s)
I20260808 13:03:13.695726 2474 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:03:13.696219 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 2474
I20260808 13:03:13.696472 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:03:13.697523 1522 external_mini_cluster.cc:1428] Setting key 6a078d25112353ad982b98a0a43ce9a9
Aug 08 13:03:13 dist-test-slave-t3q3 krb5kdc[2312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194193, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.124.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.130@KRBTEST.COM" created.
I20260808 13:03:13.727072 2667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:13.702671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:45327 (local address 127.1.124.190:42789)
0808 13:03:13.709675 (+ 7004us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:13.709684 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:13.709710 (+ 26us) server_negotiation.cc:407] Connection header received
0808 13:03:13.711184 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:13.711193 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:13.711286 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:13.711414 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:13.713065 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:13.714341 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:13.716280 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:13.717064 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:13.717219 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:13.720814 (+ 3595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:13.720821 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:13.720829 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:13.720873 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:13.723799 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:13.724616 (+ 817us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:13.724621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:13.724623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:13.724740 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:13.725292 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:13.725295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:13.725297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:13.725563 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:13.725741 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:13.726461 (+ 720us) server_negotiation.cc:299] Negotiation successful
0808 13:03:13.726703 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6852,"thread_start_us":6750,"threads_started":1}
I20260808 13:03:13.728122 2668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:13.703700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:42789 (local address 127.1.124.129:45327)
0808 13:03:13.708734 (+ 5034us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:13.708806 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:13.710896 (+ 2090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:13.711628 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:13.711637 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:13.712288 (+ 651us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:13.712874 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:13.712893 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:13.714477 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:13.714480 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:13.716142 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:13.716155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:13.717283 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:13.717288 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:13.717481 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:13.718245 (+ 764us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:13.718272 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:13.720560 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:13.723924 (+ 3364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:13.723931 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:13.723944 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:13.724485 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:13.724856 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:13.724860 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:13.724862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:13.725169 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:13.725721 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:13.725729 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:13.726067 (+ 338us) client_negotiation.cc:769] Sending connection context
0808 13:03:13.726354 (+ 287us) client_negotiation.cc:240] Negotiation successful
0808 13:03:13.726893 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4670,"thread_start_us":4410,"threads_started":1}
I20260808 13:03:13.729874 2663 heartbeater.cc:347] Connected to a master server at 127.1.124.190:42789
I20260808 13:03:13.730196 2663 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:13.731043 2663 heartbeater.cc:511] Master 127.1.124.190:42789 requested a full tablet report, sending...
I20260808 13:03:13.733404 2379 ts_manager.cc:194] Registered new tserver with Master: b2c37af441a34b67abd48bc870895267 (127.1.124.129:35363)
I20260808 13:03:13.734941 2379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:45327
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
I20260808 13:03:13.751096 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42789
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:42951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:14.095170 2675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:14.095453 2675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:14.095552 2675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:14.103024 2675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:14.103142 2675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:14.103258 2675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:03:14.113214 2675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:42951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42789
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:14.114493 2675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:14.115875 2675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:14.129184 2686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:14.129140 2687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:14.129159 2690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:14.130402 2675 server_base.cc:1034] running on GCE node
I20260808 13:03:14.131043 2675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:14.132040 2675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:14.133328 2675 hybrid_clock.cc:648] HybridClock initialized: now 1786194194133290 us; error 35 us; skew 500 ppm
I20260808 13:03:14.133669 2675 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:14 dist-test-slave-t3q3 krb5kdc[2312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194194, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:14.137140 2675 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:03:14.138707 2675 webserver.cc:460] Webserver started at http://127.1.124.130:36645/ using document root <none> and password file <none>
I20260808 13:03:14.139386 2675 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:14.139492 2675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:14.139768 2675 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:14.142872 2675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "3921439b46634ba3a315001a7bbe83b4"
format_stamp: "Formatted at 2026-08-08 13:03:14 on dist-test-slave-t3q3"
server_key: "b6521de672a507f4b0e2eee2985290ac"
server_key_iv: "a91d60f7e725dbd8a44f63110e6bb882"
server_key_version: "encryptionkey@0"
I20260808 13:03:14.143627 2675 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "3921439b46634ba3a315001a7bbe83b4"
format_stamp: "Formatted at 2026-08-08 13:03:14 on dist-test-slave-t3q3"
server_key: "b6521de672a507f4b0e2eee2985290ac"
server_key_iv: "a91d60f7e725dbd8a44f63110e6bb882"
server_key_version: "encryptionkey@0"
I20260808 13:03:14.148943 2675 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 13:03:14.152501 2700 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:14.153903 2675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 13:03:14.154042 2675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "3921439b46634ba3a315001a7bbe83b4"
format_stamp: "Formatted at 2026-08-08 13:03:14 on dist-test-slave-t3q3"
server_key: "b6521de672a507f4b0e2eee2985290ac"
server_key_iv: "a91d60f7e725dbd8a44f63110e6bb882"
server_key_version: "encryptionkey@0"
I20260808 13:03:14.154217 2675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:14.189613 2675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:14.194291 2675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:14.194540 2675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:14.195564 2675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:03:14.197144 2675 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:14.197249 2675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:14.197404 2675 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:14.197475 2675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:14.233349 2675 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:44321
I20260808 13:03:14.233362 2863 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:44321 every 8 connection(s)
I20260808 13:03:14.234867 2675 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:03:14.243374 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 2675
I20260808 13:03:14.243558 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:03:14.244194 1522 external_mini_cluster.cc:1428] Setting key 9c7837cc588f2dde9ac8c4c8b278ba86
Aug 08 13:03:14 dist-test-slave-t3q3 krb5kdc[2312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194194, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:14.257496 2866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:14.237620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:42053 (local address 127.1.124.190:42789)
0808 13:03:14.238132 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:14.238137 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:14.239002 (+ 865us) server_negotiation.cc:407] Connection header received
0808 13:03:14.240979 (+ 1977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:14.240984 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:14.241080 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:14.241203 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:14.242751 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.243546 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.246092 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.246765 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.246895 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:14.251312 (+ 4417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:14.251316 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:14.251320 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:14.251352 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:14.254222 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:14.255012 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:14.255024 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:14.255027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:14.255164 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:14.255644 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:14.255647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:14.255649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:14.255915 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:14.256109 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:14.256855 (+ 746us) server_negotiation.cc:299] Negotiation successful
0808 13:03:14.257086 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":284,"threads_started":1}
I20260808 13:03:14.258767 2867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:14.238097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:42789 (local address 127.1.124.130:42053)
0808 13:03:14.238822 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:14.238868 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:14.240685 (+ 1817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:14.241457 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:14.241466 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:14.241992 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:14.242573 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:14.242592 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.243772 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.243779 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:14.245937 (+ 2158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:14.245963 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.247030 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.247040 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:14.247378 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:14.248167 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:14.248211 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:14.251023 (+ 2812us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:14.254386 (+ 3363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:14.254397 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:14.254416 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:14.254874 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:14.255285 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:14.255290 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:14.255293 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:14.255496 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:14.256089 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:14.256099 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:14.256469 (+ 370us) client_negotiation.cc:769] Sending connection context
0808 13:03:14.256752 (+ 283us) client_negotiation.cc:240] Negotiation successful
0808 13:03:14.257352 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":275,"threads_started":1}
I20260808 13:03:14.260490 2864 heartbeater.cc:347] Connected to a master server at 127.1.124.190:42789
I20260808 13:03:14.260826 2864 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:14.261586 2864 heartbeater.cc:511] Master 127.1.124.190:42789 requested a full tablet report, sending...
I20260808 13:03:14.263206 2379 ts_manager.cc:194] Registered new tserver with Master: 3921439b46634ba3a315001a7bbe83b4 (127.1.124.130:44321)
I20260808 13:03:14.264040 2379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:42053
WARNING: no policy specified for kudu/127.1.124.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
I20260808 13:03:14.294970 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42789
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:42951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:14.635375 2873 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:14.635655 2873 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:14.635838 2873 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:14.643297 2873 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:14.643416 2873 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:14.643685 2873 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:03:14.653404 2873 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:42951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42789
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:14.654774 2873 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:14.656176 2873 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:14.669698 2887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:14.669662 2884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:14.669727 2882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:14.670545 2873 server_base.cc:1034] running on GCE node
I20260808 13:03:14.671211 2873 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:14.672324 2873 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:14.673565 2873 hybrid_clock.cc:648] HybridClock initialized: now 1786194194673517 us; error 34 us; skew 500 ppm
I20260808 13:03:14.673929 2873 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:14 dist-test-slave-t3q3 krb5kdc[2312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194194, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:14.677435 2873 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:03:14.679131 2873 webserver.cc:460] Webserver started at http://127.1.124.131:41097/ using document root <none> and password file <none>
I20260808 13:03:14.679883 2873 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:14.680051 2873 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:14.680460 2873 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:14.683557 2873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "70d4fe0bf241427c905930d070b97660"
format_stamp: "Formatted at 2026-08-08 13:03:14 on dist-test-slave-t3q3"
server_key: "98d93994e47d994fb7bad0ca86360498"
server_key_iv: "78084b777da7540de35158c762e0b462"
server_key_version: "encryptionkey@0"
I20260808 13:03:14.684543 2873 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "70d4fe0bf241427c905930d070b97660"
format_stamp: "Formatted at 2026-08-08 13:03:14 on dist-test-slave-t3q3"
server_key: "98d93994e47d994fb7bad0ca86360498"
server_key_iv: "78084b777da7540de35158c762e0b462"
server_key_version: "encryptionkey@0"
I20260808 13:03:14.690174 2873 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:03:14.694213 2901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:14.695863 2873 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:03:14.696094 2873 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "70d4fe0bf241427c905930d070b97660"
format_stamp: "Formatted at 2026-08-08 13:03:14 on dist-test-slave-t3q3"
server_key: "98d93994e47d994fb7bad0ca86360498"
server_key_iv: "78084b777da7540de35158c762e0b462"
server_key_version: "encryptionkey@0"
I20260808 13:03:14.696321 2873 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:14.716287 2873 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:14.721177 2873 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:14.721443 2873 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:14.722476 2873 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:03:14.723891 2873 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:14.723973 2873 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:14.724136 2873 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:14.724192 2873 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260808 13:03:14.738229 2663 heartbeater.cc:503] Master 127.1.124.190:42789 was elected leader, sending a full tablet report...
I20260808 13:03:14.764313 2873 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:43935
I20260808 13:03:14.764393 3078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:43935 every 8 connection(s)
I20260808 13:03:14.765695 2873 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:03:14.766506 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 2873
I20260808 13:03:14.766697 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:03:14.767396 1522 external_mini_cluster.cc:1428] Setting key b2f313bece57b3659d90fae0ac1c2eb2
Aug 08 13:03:14 dist-test-slave-t3q3 krb5kdc[2312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194194, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:14.788363 3082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:14.769360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:37839 (local address 127.1.124.190:42789)
0808 13:03:14.770379 (+ 1019us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:14.770389 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:14.772141 (+ 1752us) server_negotiation.cc:407] Connection header received
0808 13:03:14.773119 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:14.773125 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:14.773223 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:14.773349 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:14.774866 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.775678 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.777374 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.777782 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.777923 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:14.781879 (+ 3956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:14.781885 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:14.781888 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:14.781923 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:14.785222 (+ 3299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:14.786035 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:14.786040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:14.786042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:14.786187 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:14.786615 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:14.786619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:14.786621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:14.786904 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:14.787042 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:14.787779 (+ 737us) server_negotiation.cc:299] Negotiation successful
0808 13:03:14.787966 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":841,"thread_start_us":701,"threads_started":1}
I20260808 13:03:14.789361 3083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:14.769851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:42789 (local address 127.1.124.131:37839)
0808 13:03:14.770954 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:14.771011 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:14.772789 (+ 1778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:14.773599 (+ 810us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:14.773609 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:14.774127 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:14.774699 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:14.774717 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.775845 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.775850 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:14.777204 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:14.777218 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.778024 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.778033 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:14.778315 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:14.779026 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:14.779051 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:14.781639 (+ 2588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:14.785382 (+ 3743us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:14.785391 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:14.785405 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:14.785875 (+ 470us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:14.786296 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:14.786299 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:14.786302 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:14.786497 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:14.787024 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:14.787030 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:14.787403 (+ 373us) client_negotiation.cc:769] Sending connection context
0808 13:03:14.787644 (+ 241us) client_negotiation.cc:240] Negotiation successful
0808 13:03:14.788147 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"thread_start_us":542,"threads_started":1}
I20260808 13:03:14.790941 3079 heartbeater.cc:347] Connected to a master server at 127.1.124.190:42789
I20260808 13:03:14.791375 3079 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:14.792258 3079 heartbeater.cc:511] Master 127.1.124.190:42789 requested a full tablet report, sending...
I20260808 13:03:14.793835 2379 ts_manager.cc:194] Registered new tserver with Master: 70d4fe0bf241427c905930d070b97660 (127.1.124.131:43935)
I20260808 13:03:14.794708 2379 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:37839
I20260808 13:03:14.797099 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:03:14.820446 3082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:14.805099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34052 (local address 127.1.124.190:42789)
0808 13:03:14.805330 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:14.805338 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:14.805924 (+ 586us) server_negotiation.cc:407] Connection header received
0808 13:03:14.806070 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:14.806074 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:14.806160 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:14.806248 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:14.808445 (+ 2197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.809205 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.810760 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.811212 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.811312 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:14.814226 (+ 2914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:14.814230 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:14.814234 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:14.814267 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:14.817608 (+ 3341us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:14.818275 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:14.818280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:14.818282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:14.818462 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:14.818845 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:14.818848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:14.818851 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:14.819242 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:14.819439 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:14.819813 (+ 374us) server_negotiation.cc:299] Negotiation successful
0808 13:03:14.820011 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 13:03:14.830014 2379 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:03:14.838593 2379 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:03:14.869546 3100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:14.855174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:44321 (local address 127.0.0.1:43788)
0808 13:03:14.856285 (+ 1111us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:14.856346 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:14.856600 (+ 254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:14.858071 (+ 1471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:14.858083 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:14.858133 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:14.858677 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:14.858694 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.861635 (+ 2941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.861644 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:14.865091 (+ 3447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:14.865116 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.868192 (+ 3076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.868202 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:14.868526 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:14.868625 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 13:03:14.868781 (+ 156us) client_negotiation.cc:240] Negotiation successful
0808 13:03:14.869064 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"threads_started":1}
I20260808 13:03:14.871506 3102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:14.855450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43788 (local address 127.1.124.130:44321)
0808 13:03:14.857281 (+ 1831us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:14.857294 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:14.857328 (+ 34us) server_negotiation.cc:407] Connection header received
0808 13:03:14.857459 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:14.857470 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:14.857776 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:14.858933 (+ 1157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:14.859213 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.861407 (+ 2194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.865406 (+ 3999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.867866 (+ 2460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.870556 (+ 2690us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:14.870828 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:14.870913 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 13:03:14.871143 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1619,"thread_start_us":1491,"threads_started":1}
I20260808 13:03:14.875986 2763 tablet_service.cc:1467] Processing CreateTablet for tablet 3d38779e2e1d45499e6b03bafd53a01d (DEFAULT_TABLE table=test-table [id=09dffc9ffc024802b643821e8dda063d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:14.877786 2763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d38779e2e1d45499e6b03bafd53a01d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:14.878044 3100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:14.856782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:43935 (local address 127.0.0.1:47944)
0808 13:03:14.869824 (+ 13042us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:14.869849 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:14.869961 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:14.870764 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:14.870768 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:14.870795 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:14.870973 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:14.870988 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.873114 (+ 2126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.873123 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:14.875287 (+ 2164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:14.875303 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.877142 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.877148 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:14.877359 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:14.877401 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 13:03:14.877488 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 13:03:14.877659 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12920,"mutex_wait_us":25,"thread_start_us":13624,"threads_started":1}
I20260808 13:03:14.878582 3106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:14.857488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47944 (local address 127.1.124.131:43935)
0808 13:03:14.869881 (+ 12393us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:14.869889 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:14.869924 (+ 35us) server_negotiation.cc:407] Connection header received
0808 13:03:14.870196 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:14.870207 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:14.870500 (+ 293us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:14.870777 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:14.871157 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.872863 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.875444 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.876951 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.877738 (+ 787us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:14.877981 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:14.878062 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 13:03:14.878236 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12102,"thread_start_us":11938,"threads_started":1}
I20260808 13:03:14.878635 3099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:14.855320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:35363 (local address 127.0.0.1:59748)
0808 13:03:14.869638 (+ 14318us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:14.869663 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:14.870456 (+ 793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:14.871166 (+ 710us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:14.871172 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:14.871198 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:14.871555 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:14.871565 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.873345 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.873349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:14.875046 (+ 1697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:14.875060 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.877496 (+ 2436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.877502 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:14.877694 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:14.877725 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 13:03:14.877812 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 13:03:14.878302 (+ 490us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14150,"thread_start_us":442,"threads_started":1}
I20260808 13:03:14.879577 3104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:14.855950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59748 (local address 127.1.124.129:35363)
0808 13:03:14.869004 (+ 13054us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:14.869017 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:14.869779 (+ 762us) server_negotiation.cc:407] Connection header received
0808 13:03:14.870712 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:14.870722 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:14.870951 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:14.871152 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:14.871791 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.873181 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.875221 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.876943 (+ 1722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.877103 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:14.877349 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:14.878420 (+ 1071us) server_negotiation.cc:299] Negotiation successful
0808 13:03:14.878618 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12801,"thread_start_us":13081,"threads_started":1}
I20260808 13:03:14.882174 2966 tablet_service.cc:1467] Processing CreateTablet for tablet 3d38779e2e1d45499e6b03bafd53a01d (DEFAULT_TABLE table=test-table [id=09dffc9ffc024802b643821e8dda063d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:14.883661 2966 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d38779e2e1d45499e6b03bafd53a01d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:14.883183 2575 tablet_service.cc:1467] Processing CreateTablet for tablet 3d38779e2e1d45499e6b03bafd53a01d (DEFAULT_TABLE table=test-table [id=09dffc9ffc024802b643821e8dda063d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:14.884647 2575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d38779e2e1d45499e6b03bafd53a01d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:14.893010 3107 tablet_bootstrap.cc:492] T 3d38779e2e1d45499e6b03bafd53a01d P 3921439b46634ba3a315001a7bbe83b4: Bootstrap starting.
I20260808 13:03:14.896377 3108 tablet_bootstrap.cc:492] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660: Bootstrap starting.
I20260808 13:03:14.897536 3107 tablet_bootstrap.cc:654] T 3d38779e2e1d45499e6b03bafd53a01d P 3921439b46634ba3a315001a7bbe83b4: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:14.898672 3107 log.cc:824] T 3d38779e2e1d45499e6b03bafd53a01d P 3921439b46634ba3a315001a7bbe83b4: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:14.899432 3110 tablet_bootstrap.cc:492] T 3d38779e2e1d45499e6b03bafd53a01d P b2c37af441a34b67abd48bc870895267: Bootstrap starting.
I20260808 13:03:14.900274 3108 tablet_bootstrap.cc:654] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:14.901785 3108 log.cc:824] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:14.903192 3107 tablet_bootstrap.cc:492] T 3d38779e2e1d45499e6b03bafd53a01d P 3921439b46634ba3a315001a7bbe83b4: No bootstrap required, opened a new log
I20260808 13:03:14.903491 3107 ts_tablet_manager.cc:1397] T 3d38779e2e1d45499e6b03bafd53a01d P 3921439b46634ba3a315001a7bbe83b4: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260808 13:03:14.903873 3110 tablet_bootstrap.cc:654] T 3d38779e2e1d45499e6b03bafd53a01d P b2c37af441a34b67abd48bc870895267: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:14.905017 3110 log.cc:824] T 3d38779e2e1d45499e6b03bafd53a01d P b2c37af441a34b67abd48bc870895267: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:14.905892 3108 tablet_bootstrap.cc:492] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660: No bootstrap required, opened a new log
I20260808 13:03:14.906175 3108 ts_tablet_manager.cc:1397] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260808 13:03:14.907580 3110 tablet_bootstrap.cc:492] T 3d38779e2e1d45499e6b03bafd53a01d P b2c37af441a34b67abd48bc870895267: No bootstrap required, opened a new log
I20260808 13:03:14.907840 3110 ts_tablet_manager.cc:1397] T 3d38779e2e1d45499e6b03bafd53a01d P b2c37af441a34b67abd48bc870895267: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 13:03:14.915395 3107 raft_consensus.cc:348] T 3d38779e2e1d45499e6b03bafd53a01d P 3921439b46634ba3a315001a7bbe83b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3921439b46634ba3a315001a7bbe83b4" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 44321 } } peers { permanent_uuid: "b2c37af441a34b67abd48bc870895267" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35363 } } peers { permanent_uuid: "70d4fe0bf241427c905930d070b97660" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 43935 } }
I20260808 13:03:14.915920 3107 raft_consensus.cc:374] T 3d38779e2e1d45499e6b03bafd53a01d P 3921439b46634ba3a315001a7bbe83b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:14.916023 3107 raft_consensus.cc:729] T 3d38779e2e1d45499e6b03bafd53a01d P 3921439b46634ba3a315001a7bbe83b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3921439b46634ba3a315001a7bbe83b4, State: Initialized, Role: FOLLOWER
I20260808 13:03:14.915706 3110 raft_consensus.cc:348] T 3d38779e2e1d45499e6b03bafd53a01d P b2c37af441a34b67abd48bc870895267 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3921439b46634ba3a315001a7bbe83b4" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 44321 } } peers { permanent_uuid: "b2c37af441a34b67abd48bc870895267" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35363 } } peers { permanent_uuid: "70d4fe0bf241427c905930d070b97660" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 43935 } }
I20260808 13:03:14.916249 3110 raft_consensus.cc:374] T 3d38779e2e1d45499e6b03bafd53a01d P b2c37af441a34b67abd48bc870895267 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:14.916375 3110 raft_consensus.cc:729] T 3d38779e2e1d45499e6b03bafd53a01d P b2c37af441a34b67abd48bc870895267 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c37af441a34b67abd48bc870895267, State: Initialized, Role: FOLLOWER
I20260808 13:03:14.916883 3107 consensus_queue.cc:260] T 3d38779e2e1d45499e6b03bafd53a01d P 3921439b46634ba3a315001a7bbe83b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3921439b46634ba3a315001a7bbe83b4" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 44321 } } peers { permanent_uuid: "b2c37af441a34b67abd48bc870895267" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35363 } } peers { permanent_uuid: "70d4fe0bf241427c905930d070b97660" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 43935 } }
I20260808 13:03:14.916988 3110 consensus_queue.cc:260] T 3d38779e2e1d45499e6b03bafd53a01d P b2c37af441a34b67abd48bc870895267 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3921439b46634ba3a315001a7bbe83b4" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 44321 } } peers { permanent_uuid: "b2c37af441a34b67abd48bc870895267" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35363 } } peers { permanent_uuid: "70d4fe0bf241427c905930d070b97660" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 43935 } }
I20260808 13:03:14.917366 3108 raft_consensus.cc:348] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3921439b46634ba3a315001a7bbe83b4" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 44321 } } peers { permanent_uuid: "b2c37af441a34b67abd48bc870895267" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35363 } } peers { permanent_uuid: "70d4fe0bf241427c905930d070b97660" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 43935 } }
I20260808 13:03:14.917968 3108 raft_consensus.cc:374] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:14.918079 3108 raft_consensus.cc:729] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70d4fe0bf241427c905930d070b97660, State: Initialized, Role: FOLLOWER
I20260808 13:03:14.918565 2864 heartbeater.cc:503] Master 127.1.124.190:42789 was elected leader, sending a full tablet report...
I20260808 13:03:14.918767 3108 consensus_queue.cc:260] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3921439b46634ba3a315001a7bbe83b4" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 44321 } } peers { permanent_uuid: "b2c37af441a34b67abd48bc870895267" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35363 } } peers { permanent_uuid: "70d4fe0bf241427c905930d070b97660" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 43935 } }
I20260808 13:03:14.919054 3107 ts_tablet_manager.cc:1428] T 3d38779e2e1d45499e6b03bafd53a01d P 3921439b46634ba3a315001a7bbe83b4: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260808 13:03:14.920871 3110 ts_tablet_manager.cc:1428] T 3d38779e2e1d45499e6b03bafd53a01d P b2c37af441a34b67abd48bc870895267: Time spent starting tablet: real 0.013s user 0.006s sys 0.008s
I20260808 13:03:14.923074 3079 heartbeater.cc:503] Master 127.1.124.190:42789 was elected leader, sending a full tablet report...
I20260808 13:03:14.923243 3108 ts_tablet_manager.cc:1428] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660: Time spent starting tablet: real 0.017s user 0.016s sys 0.001s
I20260808 13:03:14.954989 3117 raft_consensus.cc:482] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:03:14.955436 3117 raft_consensus.cc:504] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3921439b46634ba3a315001a7bbe83b4" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 44321 } } peers { permanent_uuid: "b2c37af441a34b67abd48bc870895267" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35363 } } peers { permanent_uuid: "70d4fe0bf241427c905930d070b97660" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 43935 } }
I20260808 13:03:14.957762 3117 leader_election.cc:288] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3921439b46634ba3a315001a7bbe83b4 (127.1.124.130:44321), b2c37af441a34b67abd48bc870895267 (127.1.124.129:35363)
W20260808 13:03:14.958244 2666 tablet.cc:2369] T 3d38779e2e1d45499e6b03bafd53a01d P b2c37af441a34b67abd48bc870895267: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:03:14.965802 3102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:14.958375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:40189 (local address 127.1.124.130:44321)
0808 13:03:14.958662 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:14.958668 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:14.958693 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:03:14.958892 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:14.958901 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:14.958992 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:14.959105 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:14.959697 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.960845 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.963402 (+ 2557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.964845 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.965038 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:14.965122 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:14.965410 (+ 288us) server_negotiation.cc:299] Negotiation successful
0808 13:03:14.965544 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260808 13:03:14.965817 3083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:14.958115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:44321 (local address 127.1.124.131:40189)
0808 13:03:14.958515 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:14.958536 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:14.958657 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:14.959284 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:14.959289 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:14.959315 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:14.959551 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:14.959562 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.960990 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.960994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:14.963185 (+ 2191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:14.963206 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.965006 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.965011 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:14.965226 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:14.965267 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 13:03:14.965378 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 13:03:14.965506 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257}
I20260808 13:03:14.966032 3104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:14.958744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:44673 (local address 127.1.124.129:35363)
0808 13:03:14.958945 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:14.958951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:14.959441 (+ 490us) server_negotiation.cc:407] Connection header received
0808 13:03:14.959644 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:14.959648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:14.959725 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:14.959826 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:14.960428 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.961311 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.963690 (+ 2379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.965052 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.965206 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:14.965253 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:14.965605 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 13:03:14.965733 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 13:03:14.966502 3118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:14.958130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:35363 (local address 127.1.124.131:44673)
0808 13:03:14.959332 (+ 1202us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:14.959349 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:14.959466 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:14.959848 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:14.959851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:14.959871 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:14.960270 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:14.960280 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.961463 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.961470 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:14.963518 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:14.963528 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:14.965254 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:14.965259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:14.965435 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:14.965466 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 13:03:14.965549 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 13:03:14.965687 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1069,"mutex_wait_us":140,"thread_start_us":665,"threads_started":1}
I20260808 13:03:14.967145 2793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d38779e2e1d45499e6b03bafd53a01d" candidate_uuid: "70d4fe0bf241427c905930d070b97660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3921439b46634ba3a315001a7bbe83b4" is_pre_election: true
I20260808 13:03:14.967533 2599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d38779e2e1d45499e6b03bafd53a01d" candidate_uuid: "70d4fe0bf241427c905930d070b97660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c37af441a34b67abd48bc870895267" is_pre_election: true
I20260808 13:03:14.968021 2793 raft_consensus.cc:2454] T 3d38779e2e1d45499e6b03bafd53a01d P 3921439b46634ba3a315001a7bbe83b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70d4fe0bf241427c905930d070b97660 in term 0.
I20260808 13:03:14.968240 2599 raft_consensus.cc:2454] T 3d38779e2e1d45499e6b03bafd53a01d P b2c37af441a34b67abd48bc870895267 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70d4fe0bf241427c905930d070b97660 in term 0.
I20260808 13:03:14.969043 2904 leader_election.cc:302] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70d4fe0bf241427c905930d070b97660, b2c37af441a34b67abd48bc870895267; no voters:
I20260808 13:03:14.969547 3117 raft_consensus.cc:2781] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:03:14.969802 3117 raft_consensus.cc:482] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:03:14.970000 3117 raft_consensus.cc:3037] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:14.972734 3117 raft_consensus.cc:504] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3921439b46634ba3a315001a7bbe83b4" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 44321 } } peers { permanent_uuid: "b2c37af441a34b67abd48bc870895267" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35363 } } peers { permanent_uuid: "70d4fe0bf241427c905930d070b97660" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 43935 } }
I20260808 13:03:14.973532 3117 leader_election.cc:288] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660 [CANDIDATE]: Term 1 election: Requested vote from peers 3921439b46634ba3a315001a7bbe83b4 (127.1.124.130:44321), b2c37af441a34b67abd48bc870895267 (127.1.124.129:35363)
I20260808 13:03:14.973958 2793 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d38779e2e1d45499e6b03bafd53a01d" candidate_uuid: "70d4fe0bf241427c905930d070b97660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3921439b46634ba3a315001a7bbe83b4"
I20260808 13:03:14.974202 2793 raft_consensus.cc:3037] T 3d38779e2e1d45499e6b03bafd53a01d P 3921439b46634ba3a315001a7bbe83b4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:14.974172 2599 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d38779e2e1d45499e6b03bafd53a01d" candidate_uuid: "70d4fe0bf241427c905930d070b97660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c37af441a34b67abd48bc870895267"
I20260808 13:03:14.974380 2599 raft_consensus.cc:3037] T 3d38779e2e1d45499e6b03bafd53a01d P b2c37af441a34b67abd48bc870895267 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:14.977639 2793 raft_consensus.cc:2454] T 3d38779e2e1d45499e6b03bafd53a01d P 3921439b46634ba3a315001a7bbe83b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70d4fe0bf241427c905930d070b97660 in term 1.
I20260808 13:03:14.977656 2599 raft_consensus.cc:2454] T 3d38779e2e1d45499e6b03bafd53a01d P b2c37af441a34b67abd48bc870895267 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70d4fe0bf241427c905930d070b97660 in term 1.
I20260808 13:03:14.978088 2904 leader_election.cc:302] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 70d4fe0bf241427c905930d070b97660, b2c37af441a34b67abd48bc870895267; no voters:
I20260808 13:03:14.978410 3117 raft_consensus.cc:2781] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:14.978890 3117 raft_consensus.cc:686] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660 [term 1 LEADER]: Becoming Leader. State: Replica: 70d4fe0bf241427c905930d070b97660, State: Running, Role: LEADER
I20260808 13:03:14.979393 3117 consensus_queue.cc:237] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3921439b46634ba3a315001a7bbe83b4" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 44321 } } peers { permanent_uuid: "b2c37af441a34b67abd48bc870895267" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35363 } } peers { permanent_uuid: "70d4fe0bf241427c905930d070b97660" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 43935 } }
I20260808 13:03:14.985090 2377 catalog_manager.cc:5495] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660 reported cstate change: term changed from 0 to 1, leader changed from <none> to 70d4fe0bf241427c905930d070b97660 (127.1.124.131). New cstate: current_term: 1 leader_uuid: "70d4fe0bf241427c905930d070b97660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3921439b46634ba3a315001a7bbe83b4" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 44321 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b2c37af441a34b67abd48bc870895267" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35363 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "70d4fe0bf241427c905930d070b97660" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 43935 } health_report { overall_health: HEALTHY } } }
W20260808 13:03:14.987074 2865 tablet.cc:2369] T 3d38779e2e1d45499e6b03bafd53a01d P 3921439b46634ba3a315001a7bbe83b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 13:03:15 dist-test-slave-t3q3 krb5kdc[2312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.129@KRBTEST.COM
I20260808 13:03:15.016371 3104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:15.000683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59754 (local address 127.1.124.129:35363)
0808 13:03:15.000899 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:15.000908 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:15.001361 (+ 453us) server_negotiation.cc:407] Connection header received
0808 13:03:15.001567 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:15.001573 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:15.001661 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:15.001768 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:15.003440 (+ 1672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:15.004359 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:15.005641 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:15.006086 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:15.006224 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:15.010475 (+ 4251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:15.010491 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:15.010508 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:15.010540 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:15.013493 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:15.014285 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:15.014292 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:15.014294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:15.014500 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:15.014848 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:15.014851 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:15.014853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:15.015338 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:15.015564 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:15.015864 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 13:03:15.016025 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 13:03:15.019341 3081 tablet.cc:2369] T 3d38779e2e1d45499e6b03bafd53a01d P 70d4fe0bf241427c905930d070b97660: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 13:03:15 dist-test-slave-t3q3 krb5kdc[2312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194195, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 13:03:15 dist-test-slave-t3q3 krb5kdc[2312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194195, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.124.129@KRBTEST.COM
I20260808 13:03:15.069578 3104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:15.054959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59758 (local address 127.1.124.129:35363)
0808 13:03:15.055225 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:15.055233 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:15.055422 (+ 189us) server_negotiation.cc:407] Connection header received
0808 13:03:15.055572 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:15.055576 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:15.055646 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:15.055736 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:15.057217 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:15.057984 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:15.059268 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:15.059793 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:15.059938 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:15.063931 (+ 3993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:15.063936 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:15.063940 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:15.063973 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:15.066912 (+ 2939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:15.067586 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:15.067592 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:15.067595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:15.067732 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:15.068165 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:15.068171 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:15.068174 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:15.068445 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:15.068718 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:15.069001 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 13:03:15.069220 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
Aug 08 13:03:15 dist-test-slave-t3q3 krb5kdc[2312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 13:03:15 dist-test-slave-t3q3 krb5kdc[2312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.129@KRBTEST.COM
I20260808 13:03:15.126653 3104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:15.111284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59760 (local address 127.1.124.129:35363)
0808 13:03:15.111528 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:15.111536 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:15.111888 (+ 352us) server_negotiation.cc:407] Connection header received
0808 13:03:15.112122 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:15.112128 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:15.112313 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:15.112453 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:15.114012 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:15.114835 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:15.116277 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:15.116712 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:15.116857 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:15.121305 (+ 4448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:15.121310 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:15.121313 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:15.121349 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:15.123909 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:15.124643 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:15.124647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:15.124649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:15.124766 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:15.125127 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:15.125131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:15.125133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:15.125430 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:15.125695 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:15.126071 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 13:03:15.126323 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260808 13:03:15.128546 2539 tablet_service.cc:2908] Rejecting scan request for tablet 3d38779e2e1d45499e6b03bafd53a01d: Uninitialized: clean time has not yet been initialized
I20260808 13:03:15.132786 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 2474
I20260808 13:03:15.146458 3104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:15.133773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59766 (local address 127.1.124.129:35363)
0808 13:03:15.133974 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:15.133979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:15.134375 (+ 396us) server_negotiation.cc:407] Connection header received
0808 13:03:15.134563 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:15.134583 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:15.134672 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:15.134757 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:15.135981 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:15.136785 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:15.138169 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:15.138614 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:15.138729 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:15.141259 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:15.141263 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:15.141267 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:15.141296 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:15.144115 (+ 2819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:15.144742 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:15.144747 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:15.144749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:15.144874 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:15.145237 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:15.145241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:15.145243 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:15.145587 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:15.145804 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:15.146044 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 13:03:15.146230 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 13:03:15.147135 2656 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:15.241076 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 2474
I20260808 13:03:15.270875 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 2675
Aug 08 13:03:15 dist-test-slave-t3q3 krb5kdc[2312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.130@KRBTEST.COM
I20260808 13:03:15.287118 3102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:15.271736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43802 (local address 127.1.124.130:44321)
0808 13:03:15.271920 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:15.271925 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:15.271946 (+ 21us) server_negotiation.cc:407] Connection header received
0808 13:03:15.272145 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:15.272149 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:15.272217 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:15.272316 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:15.273613 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:15.274513 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:15.275978 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:15.276552 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:15.276678 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:15.280963 (+ 4285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:15.280981 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:15.280999 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:15.281041 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:15.284225 (+ 3184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:15.284971 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:15.284980 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:15.284983 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:15.285140 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:15.285531 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:15.285535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:15.285537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:15.285972 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:15.286248 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:15.286483 (+ 235us) server_negotiation.cc:299] Negotiation successful
0808 13:03:15.286690 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 13:03:15.287874 2852 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:15.377662 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 2675
I20260808 13:03:15.407774 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 2873
Aug 08 13:03:15 dist-test-slave-t3q3 krb5kdc[2312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.131@KRBTEST.COM
I20260808 13:03:15.428623 3140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:15.408693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47948 (local address 127.1.124.131:43935)
0808 13:03:15.409216 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:15.409221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:15.409244 (+ 23us) server_negotiation.cc:407] Connection header received
0808 13:03:15.409300 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:15.409303 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:15.409367 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:15.409494 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:15.410795 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:15.411546 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:15.412926 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:15.413368 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:15.413515 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:15.422707 (+ 9192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:15.422726 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:15.422743 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:15.422795 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:15.425981 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:15.426682 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:15.426690 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:15.426692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:15.426818 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:15.427232 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:15.427236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:15.427237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:15.427592 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:15.427817 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:15.428157 (+ 340us) server_negotiation.cc:299] Negotiation successful
0808 13:03:15.428350 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":424,"thread_start_us":272,"threads_started":1}
I20260808 13:03:15.429376 3069 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:15.542279 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 2873
I20260808 13:03:15.571874 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 2329
I20260808 13:03:15.573307 2426 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:15.683885 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 2329
2026-08-08T13:03:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4565 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:03:15 dist-test-slave-t3q3 krb5kdc[3145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:03:15 dist-test-slave-t3q3 krb5kdc[3145](info): set up 2 sockets
Aug 08 13:03:15 dist-test-slave-t3q3 krb5kdc[3145](info): commencing operation
krb5kdc: starting...
W20260808 13:03:17.000499 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.004s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:03:17 dist-test-slave-t3q3 krb5kdc[3145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:03:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:03:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:03:17.217483 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:40289
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:43715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:40289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:17.550292 3162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:17.550570 3162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:17.550777 3162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:17.557924 3162 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:03:17.558033 3162 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 13:03:17.558100 3162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:17.558202 3162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:17.558284 3162 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:03:17.558399 3162 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:03:17.568359 3162 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:43715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:40289
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:40289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:17.569717 3162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:17.571069 3162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:17.583478 3171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:17.583520 3172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:17.583570 3174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:17.584487 3162 server_base.cc:1034] running on GCE node
I20260808 13:03:17.585429 3162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:17.587006 3162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:17.588276 3162 hybrid_clock.cc:648] HybridClock initialized: now 1786194197588218 us; error 50 us; skew 500 ppm
I20260808 13:03:17.588665 3162 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:17 dist-test-slave-t3q3 krb5kdc[3145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194197, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:17.592221 3162 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:03:17.593683 3162 webserver.cc:460] Webserver started at http://127.1.124.190:46521/ using document root <none> and password file <none>
I20260808 13:03:17.594373 3162 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:17.594463 3162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:17.594753 3162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:17.597836 3162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "804012c3878543fbaf20375887aa3e86"
format_stamp: "Formatted at 2026-08-08 13:03:17 on dist-test-slave-t3q3"
server_key: "4b05fd90c0dc4a5f704b042057ebbd08"
server_key_iv: "b85010d52b8377779a919f19f6e00f41"
server_key_version: "encryptionkey@0"
I20260808 13:03:17.598650 3162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "804012c3878543fbaf20375887aa3e86"
format_stamp: "Formatted at 2026-08-08 13:03:17 on dist-test-slave-t3q3"
server_key: "4b05fd90c0dc4a5f704b042057ebbd08"
server_key_iv: "b85010d52b8377779a919f19f6e00f41"
server_key_version: "encryptionkey@0"
I20260808 13:03:17.604003 3162 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 13:03:17.607649 3188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:17.609247 3162 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:03:17.609423 3162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "804012c3878543fbaf20375887aa3e86"
format_stamp: "Formatted at 2026-08-08 13:03:17 on dist-test-slave-t3q3"
server_key: "4b05fd90c0dc4a5f704b042057ebbd08"
server_key_iv: "b85010d52b8377779a919f19f6e00f41"
server_key_version: "encryptionkey@0"
I20260808 13:03:17.609651 3162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:17.629756 3162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:17.634166 3162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:17.634462 3162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:17.658115 3162 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:40289
I20260808 13:03:17.658154 3272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:40289 every 8 connection(s)
I20260808 13:03:17.659663 3162 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:03:17.663552 3274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:17.669760 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 3162
I20260808 13:03:17.669988 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:03:17.670890 1522 external_mini_cluster.cc:1428] Setting key 612fd7baeaf660755a612e0a7dc19722
I20260808 13:03:17.675062 3274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86: Bootstrap starting.
I20260808 13:03:17.678893 3274 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:17.679947 3274 log.cc:824] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:17.683019 3274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86: No bootstrap required, opened a new log
Aug 08 13:03:17 dist-test-slave-t3q3 krb5kdc[3145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194197, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:17.691794 3274 raft_consensus.cc:348] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "804012c3878543fbaf20375887aa3e86" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40289 } }
I20260808 13:03:17.692253 3274 raft_consensus.cc:374] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:17.692380 3274 raft_consensus.cc:729] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 804012c3878543fbaf20375887aa3e86, State: Initialized, Role: FOLLOWER
I20260808 13:03:17.693042 3274 consensus_queue.cc:260] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "804012c3878543fbaf20375887aa3e86" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40289 } }
I20260808 13:03:17.693296 3274 raft_consensus.cc:388] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:03:17.693450 3274 raft_consensus.cc:482] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:03:17.693681 3274 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:17.694953 3279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:17.673151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57382 (local address 127.1.124.190:40289)
0808 13:03:17.674001 (+ 850us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:17.674013 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:17.674056 (+ 43us) server_negotiation.cc:407] Connection header received
0808 13:03:17.675850 (+ 1794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:17.675873 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:17.676336 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:17.676668 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:17.678557 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:17.679932 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:17.681519 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:17.682199 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:17.682338 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:17.687188 (+ 4850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:17.687196 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:17.687211 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:17.687246 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:17.690635 (+ 3389us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:17.691349 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:17.691356 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:17.691362 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:17.691529 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:17.691934 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:17.691938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:17.691940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:17.692690 (+ 750us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:17.693046 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:17.693476 (+ 430us) server_negotiation.cc:299] Negotiation successful
0808 13:03:17.693927 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":413,"threads_started":1}
I20260808 13:03:17.696308 3274 raft_consensus.cc:504] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "804012c3878543fbaf20375887aa3e86" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40289 } }
I20260808 13:03:17.696861 3274 leader_election.cc:302] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 804012c3878543fbaf20375887aa3e86; no voters:
I20260808 13:03:17.697448 3274 leader_election.cc:288] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:03:17.697644 3282 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:17.698441 3282 raft_consensus.cc:686] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86 [term 1 LEADER]: Becoming Leader. State: Replica: 804012c3878543fbaf20375887aa3e86, State: Running, Role: LEADER
I20260808 13:03:17.699000 3282 consensus_queue.cc:237] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "804012c3878543fbaf20375887aa3e86" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40289 } }
I20260808 13:03:17.699690 3274 sys_catalog.cc:564] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:03:17.702718 3284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "804012c3878543fbaf20375887aa3e86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "804012c3878543fbaf20375887aa3e86" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40289 } } }
I20260808 13:03:17.702744 3285 sys_catalog.cc:455] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 804012c3878543fbaf20375887aa3e86. Latest consensus state: current_term: 1 leader_uuid: "804012c3878543fbaf20375887aa3e86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "804012c3878543fbaf20375887aa3e86" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40289 } } }
I20260808 13:03:17.703163 3284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86 [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:17.703296 3285 sys_catalog.cc:458] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86 [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:17.708637 3292 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:03:17.714617 3292 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:03:17.723632 3292 catalog_manager.cc:1347] Generated new cluster ID: d2f4a25cc15941029365e5df4ab64099
I20260808 13:03:17.723815 3292 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:03:17.746400 3292 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:03:17.747773 3292 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:03:17.756141 3292 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86: Generated new TSK 0
I20260808 13:03:17.757004 3292 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.124.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
I20260808 13:03:17.815997 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:40289
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:43715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:18.162374 3313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:18.162684 3313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:18.162786 3313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:18.170343 3313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:18.170469 3313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:18.170560 3313 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 13:03:18.170650 3313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:03:18.181545 3313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:43715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.124.190:40289
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:18.182830 3313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:18.184214 3313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:18.198019 3320 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:18.197989 3321 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:18.198141 3324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:18.198726 3313 server_base.cc:1034] running on GCE node
I20260808 13:03:18.199412 3313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:18.200513 3313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:18.201746 3313 hybrid_clock.cc:648] HybridClock initialized: now 1786194198201679 us; error 51 us; skew 500 ppm
I20260808 13:03:18.202090 3313 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:18 dist-test-slave-t3q3 krb5kdc[3145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194198, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:18.205880 3313 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:03:18.207311 3313 webserver.cc:460] Webserver started at http://127.1.124.129:35441/ using document root <none> and password file <none>
I20260808 13:03:18.208000 3313 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:18.208170 3313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:18.208511 3313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:18.211617 3313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "5dea7daa69ce4fe2913c2e44e407469c"
format_stamp: "Formatted at 2026-08-08 13:03:18 on dist-test-slave-t3q3"
server_key: "851752747db73f817889b5da74b0f295"
server_key_iv: "1c98cd5c4439535bed13d17e54798bdd"
server_key_version: "encryptionkey@0"
I20260808 13:03:18.212514 3313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "5dea7daa69ce4fe2913c2e44e407469c"
format_stamp: "Formatted at 2026-08-08 13:03:18 on dist-test-slave-t3q3"
server_key: "851752747db73f817889b5da74b0f295"
server_key_iv: "1c98cd5c4439535bed13d17e54798bdd"
server_key_version: "encryptionkey@0"
I20260808 13:03:18.217941 3313 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:03:18.221563 3333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:18.222883 3313 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260808 13:03:18.223059 3313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "5dea7daa69ce4fe2913c2e44e407469c"
format_stamp: "Formatted at 2026-08-08 13:03:18 on dist-test-slave-t3q3"
server_key: "851752747db73f817889b5da74b0f295"
server_key_iv: "1c98cd5c4439535bed13d17e54798bdd"
server_key_version: "encryptionkey@0"
I20260808 13:03:18.223307 3313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:18.250572 3313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:18.255035 3313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:18.255366 3313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:18.258487 3313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:18.258601 3313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:18.258759 3313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:18.258837 3313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 13:03:18 dist-test-slave-t3q3 krb5kdc[3145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194198, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:18.283387 3344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:18.259272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:42647 (local address 127.1.124.190:40289)
0808 13:03:18.259858 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:18.259865 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:18.261131 (+ 1266us) server_negotiation.cc:407] Connection header received
0808 13:03:18.263179 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:18.263189 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:18.263285 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:18.263438 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:18.266111 (+ 2673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:18.267210 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:18.269995 (+ 2785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:18.270446 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:18.270604 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:18.275248 (+ 4644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:18.275253 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:18.275256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:18.275288 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:18.279411 (+ 4123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:18.280642 (+ 1231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:18.280647 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:18.280651 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:18.280776 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:18.281427 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:18.281433 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:18.281436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:18.281836 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:18.281997 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:18.282767 (+ 770us) server_negotiation.cc:299] Negotiation successful
0808 13:03:18.283000 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":279,"threads_started":1}
I20260808 13:03:18.284646 3345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:18.259968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:40289 (local address 127.1.124.129:42647)
0808 13:03:18.260932 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:18.260995 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:18.262897 (+ 1902us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:18.263829 (+ 932us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:18.263847 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:18.264774 (+ 927us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:18.265804 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:18.265840 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:18.267395 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:18.267406 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:18.269822 (+ 2416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:18.269835 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:18.270709 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:18.270718 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:18.270953 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:18.271685 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:18.271708 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:18.274935 (+ 3227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:18.279656 (+ 4721us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:18.279668 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:18.279695 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:18.280509 (+ 814us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:18.280967 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:18.280974 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:18.280980 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:18.281289 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:18.281988 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:18.281996 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:18.282372 (+ 376us) client_negotiation.cc:769] Sending connection context
0808 13:03:18.282651 (+ 279us) client_negotiation.cc:240] Negotiation successful
0808 13:03:18.283190 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"thread_start_us":368,"threads_started":1}
I20260808 13:03:18.308876 3344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:18.291477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:55189 (local address 127.1.124.190:40289)
0808 13:03:18.291650 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:18.291654 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:18.292489 (+ 835us) server_negotiation.cc:407] Connection header received
0808 13:03:18.292676 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:18.292684 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:18.292788 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:18.292920 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:18.293902 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:18.295343 (+ 1441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:18.297261 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:18.297878 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:18.297957 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:18.301086 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:18.301095 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:18.301100 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:18.301136 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:18.305020 (+ 3884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:18.306032 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:18.306040 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:18.306044 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:18.306223 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:18.306811 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:18.306819 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:18.306824 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:18.307255 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:18.307433 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:18.308248 (+ 815us) server_negotiation.cc:299] Negotiation successful
0808 13:03:18.308485 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 13:03:18.308995 3457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:18.291592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:40289 (local address 127.1.124.129:55189)
0808 13:03:18.292351 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:18.292377 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:18.292542 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:18.292950 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:18.292954 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:18.293462 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:18.293753 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:18.293762 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:18.295532 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:18.295538 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:18.297035 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:18.297057 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:18.298496 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:18.298522 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:18.298750 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:18.299510 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:18.299543 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:18.300837 (+ 1294us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:18.305251 (+ 4414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:18.305260 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:18.305266 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:18.305866 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:18.306392 (+ 526us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:18.306398 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:18.306407 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:18.306622 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:18.307525 (+ 903us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:18.307532 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:18.307869 (+ 337us) client_negotiation.cc:769] Sending connection context
0808 13:03:18.308130 (+ 261us) client_negotiation.cc:240] Negotiation successful
0808 13:03:18.308527 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":346,"threads_started":1}
I20260808 13:03:18.311369 3313 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:46485
I20260808 13:03:18.311378 3516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:46485 every 8 connection(s)
I20260808 13:03:18.313042 3313 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:03:18.315153 3517 heartbeater.cc:347] Connected to a master server at 127.1.124.190:40289
I20260808 13:03:18.315342 3517 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:18.315917 3517 heartbeater.cc:511] Master 127.1.124.190:40289 requested a full tablet report, sending...
I20260808 13:03:18.317827 3209 ts_manager.cc:194] Registered new tserver with Master: 5dea7daa69ce4fe2913c2e44e407469c (127.1.124.129:46485)
I20260808 13:03:18.319235 3209 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:42647
I20260808 13:03:18.319355 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 3313
I20260808 13:03:18.319578 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:03:18.320369 1522 external_mini_cluster.cc:1428] Setting key af3d785e579d15ab52a39ff05e9ad8bf
WARNING: no policy specified for kudu/127.1.124.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
I20260808 13:03:18.364851 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:40289
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:43715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:18.697909 3521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:18.698267 3521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:18.698446 3521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:18.705760 3521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:18.705914 3521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:18.706074 3521 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 13:03:18.706223 3521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:03:18.719715 3521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:43715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.124.190:40289
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:18.722043 3521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:18.723922 3521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:18.741762 3528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:18.741732 3536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:18.741796 3530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:18.743050 3521 server_base.cc:1034] running on GCE node
I20260808 13:03:18.743872 3521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:18.744938 3521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:18.746186 3521 hybrid_clock.cc:648] HybridClock initialized: now 1786194198746123 us; error 35 us; skew 500 ppm
I20260808 13:03:18.746553 3521 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:18 dist-test-slave-t3q3 krb5kdc[3145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194198, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:18.750023 3521 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:03:18.751691 3521 webserver.cc:460] Webserver started at http://127.1.124.130:46323/ using document root <none> and password file <none>
I20260808 13:03:18.752588 3521 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:18.752736 3521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:18.753096 3521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:18.756047 3521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd"
format_stamp: "Formatted at 2026-08-08 13:03:18 on dist-test-slave-t3q3"
server_key: "5a17ed3deaa259e5e1013575cf6a7939"
server_key_iv: "a3a8393bcc2f2f779947bb156d443d78"
server_key_version: "encryptionkey@0"
I20260808 13:03:18.756953 3521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd"
format_stamp: "Formatted at 2026-08-08 13:03:18 on dist-test-slave-t3q3"
server_key: "5a17ed3deaa259e5e1013575cf6a7939"
server_key_iv: "a3a8393bcc2f2f779947bb156d443d78"
server_key_version: "encryptionkey@0"
I20260808 13:03:18.762323 3521 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:03:18.765928 3546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:18.767305 3521 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:03:18.767493 3521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd"
format_stamp: "Formatted at 2026-08-08 13:03:18 on dist-test-slave-t3q3"
server_key: "5a17ed3deaa259e5e1013575cf6a7939"
server_key_iv: "a3a8393bcc2f2f779947bb156d443d78"
server_key_version: "encryptionkey@0"
I20260808 13:03:18.767737 3521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:18.787595 3521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:18.792047 3521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:18.792425 3521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:18.795322 3521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:18.795434 3521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:18.795571 3521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:18.795644 3521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 13:03:18 dist-test-slave-t3q3 krb5kdc[3145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194198, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:18.820626 3344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:18.796047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:58901 (local address 127.1.124.190:40289)
0808 13:03:18.796288 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:18.796295 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:18.797770 (+ 1475us) server_negotiation.cc:407] Connection header received
0808 13:03:18.799804 (+ 2034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:18.799810 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:18.799898 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:18.800023 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:18.802609 (+ 2586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:18.803942 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:18.806724 (+ 2782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:18.807156 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:18.807301 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:18.812676 (+ 5375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:18.812689 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:18.812692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:18.812726 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:18.816184 (+ 3458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:18.817284 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:18.817290 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:18.817294 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:18.817424 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:18.818071 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:18.818075 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:18.818077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:18.818464 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:18.818636 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:18.819772 (+ 1136us) server_negotiation.cc:299] Negotiation successful
0808 13:03:18.820136 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 13:03:18.821537 3560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:18.796742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:40289 (local address 127.1.124.130:58901)
0808 13:03:18.797555 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:18.797601 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:18.799515 (+ 1914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:18.800383 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:18.800398 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:18.801291 (+ 893us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:18.802301 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:18.802325 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:18.804159 (+ 1834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:18.804168 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:18.806479 (+ 2311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:18.806492 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:18.807411 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:18.807419 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:18.807764 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:18.808704 (+ 940us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:18.808741 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:18.812414 (+ 3673us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:18.816392 (+ 3978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:18.816401 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:18.816417 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:18.817146 (+ 729us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:18.817543 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:18.817552 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:18.817556 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:18.817898 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:18.818705 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:18.818711 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:18.819268 (+ 557us) client_negotiation.cc:769] Sending connection context
0808 13:03:18.819601 (+ 333us) client_negotiation.cc:240] Negotiation successful
0808 13:03:18.820187 (+ 586us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":348,"threads_started":1}
I20260808 13:03:18.843102 3344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:18.827305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:46675 (local address 127.1.124.190:40289)
0808 13:03:18.827515 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:18.827523 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:18.828367 (+ 844us) server_negotiation.cc:407] Connection header received
0808 13:03:18.828617 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:18.828623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:18.828720 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:18.828832 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:18.830205 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:18.831249 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:18.833456 (+ 2207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:18.833875 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:18.834009 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:18.835902 (+ 1893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:18.835908 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:18.835913 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:18.835967 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:18.839807 (+ 3840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:18.840746 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:18.840752 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:18.840756 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:18.840939 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:18.841414 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:18.841419 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:18.841422 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:18.841936 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:18.842104 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:18.842506 (+ 402us) server_negotiation.cc:299] Negotiation successful
0808 13:03:18.842723 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 13:03:18.843202 3658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:18.827378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:40289 (local address 127.1.124.130:46675)
0808 13:03:18.828225 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:18.828257 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:18.828438 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:18.829044 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:18.829047 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:18.829535 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:18.830056 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:18.830075 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:18.831405 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:18.831410 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:18.833305 (+ 1895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:18.833323 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:18.834077 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:18.834083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:18.834249 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:18.834925 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:18.834942 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:18.835716 (+ 774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:18.839974 (+ 4258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:18.839982 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:18.839988 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:18.840616 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:18.841088 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:18.841096 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:18.841102 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:18.841287 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:18.842063 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:18.842068 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:18.842264 (+ 196us) client_negotiation.cc:769] Sending connection context
0808 13:03:18.842398 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 13:03:18.842740 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":394,"threads_started":1}
I20260808 13:03:18.851190 3521 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:36257
I20260808 13:03:18.851233 3728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:36257 every 8 connection(s)
I20260808 13:03:18.852506 3521 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:03:18.854672 3729 heartbeater.cc:347] Connected to a master server at 127.1.124.190:40289
I20260808 13:03:18.854914 3729 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:18.855564 3729 heartbeater.cc:511] Master 127.1.124.190:40289 requested a full tablet report, sending...
I20260808 13:03:18.856987 3209 ts_manager.cc:194] Registered new tserver with Master: ef2eb4eb991b40f98a5ff14e6a2da2cd (127.1.124.130:36257)
I20260808 13:03:18.857775 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 3521
I20260808 13:03:18.857949 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:03:18.858086 3209 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:58901
I20260808 13:03:18.858599 1522 external_mini_cluster.cc:1428] Setting key 703dc717c08873cfcb2b1f5fe5405313
WARNING: no policy specified for kudu/127.1.124.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
I20260808 13:03:18.905229 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:40289
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:43715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:19.240494 3734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:19.240793 3734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:19.240971 3734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:19.248164 3734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:19.248267 3734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:19.248454 3734 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 13:03:19.248574 3734 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:03:19.258694 3734 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:43715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.124.190:40289
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:19.260028 3734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:19.261404 3734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:19.274936 3745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:19.274899 3743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:19.274986 3747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:19.275748 3734 server_base.cc:1034] running on GCE node
I20260808 13:03:19.276502 3734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:19.277400 3734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:19.278641 3734 hybrid_clock.cc:648] HybridClock initialized: now 1786194199278560 us; error 53 us; skew 500 ppm
I20260808 13:03:19.278962 3734 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:19 dist-test-slave-t3q3 krb5kdc[3145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194199, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:19.282449 3734 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:03:19.284135 3734 webserver.cc:460] Webserver started at http://127.1.124.131:38185/ using document root <none> and password file <none>
I20260808 13:03:19.284847 3734 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:19.284982 3734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:19.285315 3734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:19.288429 3734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "b5eef982d5114c18a141bbbd703b44c4"
format_stamp: "Formatted at 2026-08-08 13:03:19 on dist-test-slave-t3q3"
server_key: "af0d7246b3b21486e031d564d15eded2"
server_key_iv: "f849d71bf68165e2dfa7e3a9188644db"
server_key_version: "encryptionkey@0"
I20260808 13:03:19.289245 3734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "b5eef982d5114c18a141bbbd703b44c4"
format_stamp: "Formatted at 2026-08-08 13:03:19 on dist-test-slave-t3q3"
server_key: "af0d7246b3b21486e031d564d15eded2"
server_key_iv: "f849d71bf68165e2dfa7e3a9188644db"
server_key_version: "encryptionkey@0"
I20260808 13:03:19.294378 3734 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:03:19.297873 3755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:19.299199 3734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:03:19.299378 3734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "b5eef982d5114c18a141bbbd703b44c4"
format_stamp: "Formatted at 2026-08-08 13:03:19 on dist-test-slave-t3q3"
server_key: "af0d7246b3b21486e031d564d15eded2"
server_key_iv: "f849d71bf68165e2dfa7e3a9188644db"
server_key_version: "encryptionkey@0"
I20260808 13:03:19.299607 3734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:19.314013 3734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:19.318552 3734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:19.318835 3734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:19.321563 3734 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:19.321645 3734 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:19.321817 3734 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:19.321908 3734 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:19.322458 3517 heartbeater.cc:503] Master 127.1.124.190:40289 was elected leader, sending a full tablet report...
Aug 08 13:03:19 dist-test-slave-t3q3 krb5kdc[3145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194199, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:19.347597 3344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:19.322084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:45629 (local address 127.1.124.190:40289)
0808 13:03:19.322304 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:19.322310 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:19.324010 (+ 1700us) server_negotiation.cc:407] Connection header received
0808 13:03:19.327167 (+ 3157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:19.327175 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:19.327264 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:19.327381 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:19.330096 (+ 2715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.331410 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.334027 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.334634 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.334789 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:19.339794 (+ 5005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:19.339800 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:19.339824 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:19.339881 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:19.343522 (+ 3641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:19.344724 (+ 1202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:19.344732 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:19.344736 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:19.344879 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:19.345548 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:19.345554 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:19.345560 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:19.345849 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:19.345970 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:19.347119 (+ 1149us) server_negotiation.cc:299] Negotiation successful
0808 13:03:19.347298 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 13:03:19.349823 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:19.322738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:40289 (local address 127.1.124.131:45629)
0808 13:03:19.323832 (+ 1094us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:19.323896 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:19.326830 (+ 2934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:19.327818 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:19.327829 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:19.328849 (+ 1020us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:19.329824 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:19.329886 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.331572 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.331578 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:19.333849 (+ 2271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:19.333871 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.334893 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.334901 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:19.335138 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:19.335949 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:19.335973 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:19.339513 (+ 3540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:19.343687 (+ 4174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:19.343700 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:19.343723 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:19.344569 (+ 846us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:19.345004 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:19.345013 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:19.345019 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:19.345353 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:19.345997 (+ 644us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:19.346010 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:19.346601 (+ 591us) client_negotiation.cc:769] Sending connection context
0808 13:03:19.346977 (+ 376us) client_negotiation.cc:240] Negotiation successful
0808 13:03:19.347704 (+ 727us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":666,"thread_start_us":418,"threads_started":1}
I20260808 13:03:19.374472 3877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:19.358935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:40289 (local address 127.1.124.131:38099)
0808 13:03:19.359569 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:19.359585 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:19.359755 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:19.360125 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:19.360131 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:19.360613 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:19.360902 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:19.360911 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.361920 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.361926 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:19.363292 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:19.363315 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.364541 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.364554 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:19.364783 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:19.365666 (+ 883us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:19.365694 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:19.367010 (+ 1316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:19.371074 (+ 4064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:19.371081 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:19.371091 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:19.371841 (+ 750us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:19.372394 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:19.372401 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:19.372410 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:19.372615 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:19.373391 (+ 776us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:19.373395 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:19.373571 (+ 176us) client_negotiation.cc:769] Sending connection context
0808 13:03:19.373749 (+ 178us) client_negotiation.cc:240] Negotiation successful
0808 13:03:19.374143 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":373,"threads_started":1}
I20260808 13:03:19.375187 3344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:19.358923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:38099 (local address 127.1.124.190:40289)
0808 13:03:19.359101 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:19.359105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:19.359679 (+ 574us) server_negotiation.cc:407] Connection header received
0808 13:03:19.359878 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:19.359883 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:19.359958 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:19.360061 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:19.361046 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.361775 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.363467 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.364309 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.364480 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:19.367275 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:19.367284 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:19.367290 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:19.367340 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:19.370946 (+ 3606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:19.372036 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:19.372041 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:19.372044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:19.372245 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:19.372734 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:19.372739 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:19.372743 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:19.373157 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:19.373307 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:19.374784 (+ 1477us) server_negotiation.cc:299] Negotiation successful
0808 13:03:19.374950 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 13:03:19.377149 3734 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:39845
I20260808 13:03:19.377171 3935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:39845 every 8 connection(s)
I20260808 13:03:19.378822 3734 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:03:19.380844 3937 heartbeater.cc:347] Connected to a master server at 127.1.124.190:40289
I20260808 13:03:19.381070 3937 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:19.381695 3937 heartbeater.cc:511] Master 127.1.124.190:40289 requested a full tablet report, sending...
I20260808 13:03:19.382944 3209 ts_manager.cc:194] Registered new tserver with Master: b5eef982d5114c18a141bbbd703b44c4 (127.1.124.131:39845)
I20260808 13:03:19.383751 3209 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:45629
I20260808 13:03:19.386662 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 3734
I20260808 13:03:19.386845 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:03:19.387411 1522 external_mini_cluster.cc:1428] Setting key 8527586c99983eacca1bff4efb74f4f8
I20260808 13:03:19.390421 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 13:03:19 dist-test-slave-t3q3 krb5kdc[3145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194199, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 13:03:19 dist-test-slave-t3q3 krb5kdc[3145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194199, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:19.442731 3344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:19.429016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41484 (local address 127.1.124.190:40289)
0808 13:03:19.429257 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:19.429265 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:19.429501 (+ 236us) server_negotiation.cc:407] Connection header received
0808 13:03:19.429728 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:19.429732 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:19.429808 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:19.429926 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:19.431355 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.432175 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.433521 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.433955 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.434097 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:19.438036 (+ 3939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:19.438041 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:19.438045 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:19.438076 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:19.440528 (+ 2452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:19.441123 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:19.441127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:19.441129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:19.441254 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:19.441594 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:19.441598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:19.441599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:19.441858 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:19.442033 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:19.442286 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 13:03:19.442443 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 13:03:19.446700 3209 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41484:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:03:19.453531 3209 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:03:19.493149 3964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:19.465799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40702 (local address 127.1.124.131:39845)
0808 13:03:19.480992 (+ 15193us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:19.481002 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:19.481233 (+ 231us) server_negotiation.cc:407] Connection header received
0808 13:03:19.481652 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:19.481664 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:19.481983 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:19.482221 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:19.483518 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.485761 (+ 2243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.489405 (+ 3644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.491395 (+ 1990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.491556 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:19.491829 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:19.492585 (+ 756us) server_negotiation.cc:299] Negotiation successful
0808 13:03:19.492754 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15003,"thread_start_us":15198,"threads_started":1}
I20260808 13:03:19.493760 3962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:19.465448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:39845 (local address 127.0.0.1:40702)
0808 13:03:19.481062 (+ 15614us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:19.481112 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:19.481313 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:19.482632 (+ 1319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:19.482643 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:19.482685 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:19.483211 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:19.483227 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.485980 (+ 2753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.485988 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:19.489158 (+ 3170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:19.489179 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.491977 (+ 2798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.491982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:19.492213 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:19.492273 (+ 60us) client_negotiation.cc:769] Sending connection context
0808 13:03:19.492408 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 13:03:19.493346 (+ 938us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15363,"mutex_wait_us":658,"thread_start_us":15220,"threads_started":1}
I20260808 13:03:19.497726 3828 tablet_service.cc:1467] Processing CreateTablet for tablet 269b4cb903c14f889f57c4c42a740e1f (DEFAULT_TABLE table=test-table [id=dc96a4a6ea2142f79fd4d85017980ca6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:19.499265 3968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:19.466690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57456 (local address 127.1.124.130:36257)
0808 13:03:19.491640 (+ 24950us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:19.491651 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:19.492104 (+ 453us) server_negotiation.cc:407] Connection header received
0808 13:03:19.492289 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:19.492296 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:19.492522 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:19.492709 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:19.493612 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.494996 (+ 1384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.496918 (+ 1922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.498224 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.498380 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:19.498585 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:19.498815 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 13:03:19.498965 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24788,"thread_start_us":24669,"threads_started":1}
I20260808 13:03:19.499264 3963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:19.466101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:36257 (local address 127.0.0.1:57456)
0808 13:03:19.491936 (+ 25835us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:19.491953 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:19.492116 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:19.492918 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:19.492924 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:19.492950 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:19.493382 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:19.493395 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.495141 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.495146 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:19.496762 (+ 1616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:19.496775 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.498429 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.498436 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:19.498619 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:19.498654 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 13:03:19.498761 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 13:03:19.498886 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25741,"mutex_wait_us":68,"thread_start_us":23564,"threads_started":1}
I20260808 13:03:19.499341 3828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 269b4cb903c14f889f57c4c42a740e1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:19.500174 3967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:19.465844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:46485 (local address 127.0.0.1:36668)
0808 13:03:19.489969 (+ 24125us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:19.489993 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:19.490159 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:19.492278 (+ 2119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:19.492282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:19.492304 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:19.492836 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:19.492849 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.494898 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.494905 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:19.497197 (+ 2292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:19.497212 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.499347 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.499355 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:19.499513 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:19.499543 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 13:03:19.499635 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 13:03:19.499759 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23959,"mutex_wait_us":7,"thread_start_us":25882,"threads_started":1}
I20260808 13:03:19.500285 3965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:19.466102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36668 (local address 127.1.124.129:46485)
0808 13:03:19.489472 (+ 23370us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:19.489484 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:19.490316 (+ 832us) server_negotiation.cc:407] Connection header received
0808 13:03:19.490448 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:19.490459 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:19.490756 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:19.490983 (+ 227us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:19.493134 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.494756 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.497387 (+ 2631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.498851 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.498992 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:19.499191 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:19.499725 (+ 534us) server_negotiation.cc:299] Negotiation successful
0808 13:03:19.499926 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23180,"thread_start_us":22967,"threads_started":1}
I20260808 13:03:19.503525 3408 tablet_service.cc:1467] Processing CreateTablet for tablet 269b4cb903c14f889f57c4c42a740e1f (DEFAULT_TABLE table=test-table [id=dc96a4a6ea2142f79fd4d85017980ca6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:19.503856 3623 tablet_service.cc:1467] Processing CreateTablet for tablet 269b4cb903c14f889f57c4c42a740e1f (DEFAULT_TABLE table=test-table [id=dc96a4a6ea2142f79fd4d85017980ca6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:19.504937 3408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 269b4cb903c14f889f57c4c42a740e1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:19.505210 3623 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 269b4cb903c14f889f57c4c42a740e1f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:19.514505 3970 tablet_bootstrap.cc:492] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4: Bootstrap starting.
I20260808 13:03:19.516561 3971 tablet_bootstrap.cc:492] T 269b4cb903c14f889f57c4c42a740e1f P 5dea7daa69ce4fe2913c2e44e407469c: Bootstrap starting.
I20260808 13:03:19.518106 3970 tablet_bootstrap.cc:654] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:19.519404 3970 log.cc:824] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:19.520823 3971 tablet_bootstrap.cc:654] T 269b4cb903c14f889f57c4c42a740e1f P 5dea7daa69ce4fe2913c2e44e407469c: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:19.522037 3971 log.cc:824] T 269b4cb903c14f889f57c4c42a740e1f P 5dea7daa69ce4fe2913c2e44e407469c: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:19.522701 3970 tablet_bootstrap.cc:492] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4: No bootstrap required, opened a new log
I20260808 13:03:19.522989 3970 ts_tablet_manager.cc:1397] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.001s
I20260808 13:03:19.521045 3972 tablet_bootstrap.cc:492] T 269b4cb903c14f889f57c4c42a740e1f P ef2eb4eb991b40f98a5ff14e6a2da2cd: Bootstrap starting.
I20260808 13:03:19.526024 3971 tablet_bootstrap.cc:492] T 269b4cb903c14f889f57c4c42a740e1f P 5dea7daa69ce4fe2913c2e44e407469c: No bootstrap required, opened a new log
I20260808 13:03:19.526300 3971 ts_tablet_manager.cc:1397] T 269b4cb903c14f889f57c4c42a740e1f P 5dea7daa69ce4fe2913c2e44e407469c: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260808 13:03:19.526803 3972 tablet_bootstrap.cc:654] T 269b4cb903c14f889f57c4c42a740e1f P ef2eb4eb991b40f98a5ff14e6a2da2cd: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:19.528311 3972 log.cc:824] T 269b4cb903c14f889f57c4c42a740e1f P ef2eb4eb991b40f98a5ff14e6a2da2cd: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:19.532553 3972 tablet_bootstrap.cc:492] T 269b4cb903c14f889f57c4c42a740e1f P ef2eb4eb991b40f98a5ff14e6a2da2cd: No bootstrap required, opened a new log
I20260808 13:03:19.532950 3972 ts_tablet_manager.cc:1397] T 269b4cb903c14f889f57c4c42a740e1f P ef2eb4eb991b40f98a5ff14e6a2da2cd: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260808 13:03:19.532630 3970 raft_consensus.cc:348] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } } peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } }
I20260808 13:03:19.533181 3970 raft_consensus.cc:374] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:19.533385 3970 raft_consensus.cc:729] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5eef982d5114c18a141bbbd703b44c4, State: Initialized, Role: FOLLOWER
I20260808 13:03:19.534128 3970 consensus_queue.cc:260] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } } peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } }
I20260808 13:03:19.535336 3937 heartbeater.cc:503] Master 127.1.124.190:40289 was elected leader, sending a full tablet report...
I20260808 13:03:19.535712 3971 raft_consensus.cc:348] T 269b4cb903c14f889f57c4c42a740e1f P 5dea7daa69ce4fe2913c2e44e407469c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } } peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } }
I20260808 13:03:19.536106 3970 ts_tablet_manager.cc:1428] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4: Time spent starting tablet: real 0.013s user 0.010s sys 0.001s
I20260808 13:03:19.536139 3971 raft_consensus.cc:374] T 269b4cb903c14f889f57c4c42a740e1f P 5dea7daa69ce4fe2913c2e44e407469c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:19.536238 3971 raft_consensus.cc:729] T 269b4cb903c14f889f57c4c42a740e1f P 5dea7daa69ce4fe2913c2e44e407469c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dea7daa69ce4fe2913c2e44e407469c, State: Initialized, Role: FOLLOWER
I20260808 13:03:19.536968 3971 consensus_queue.cc:260] T 269b4cb903c14f889f57c4c42a740e1f P 5dea7daa69ce4fe2913c2e44e407469c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } } peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } }
I20260808 13:03:19.538688 3971 ts_tablet_manager.cc:1428] T 269b4cb903c14f889f57c4c42a740e1f P 5dea7daa69ce4fe2913c2e44e407469c: Time spent starting tablet: real 0.012s user 0.009s sys 0.005s
I20260808 13:03:19.543807 3972 raft_consensus.cc:348] T 269b4cb903c14f889f57c4c42a740e1f P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } } peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } }
I20260808 13:03:19.544203 3972 raft_consensus.cc:374] T 269b4cb903c14f889f57c4c42a740e1f P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:19.544320 3972 raft_consensus.cc:729] T 269b4cb903c14f889f57c4c42a740e1f P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2eb4eb991b40f98a5ff14e6a2da2cd, State: Initialized, Role: FOLLOWER
I20260808 13:03:19.544829 3972 consensus_queue.cc:260] T 269b4cb903c14f889f57c4c42a740e1f P ef2eb4eb991b40f98a5ff14e6a2da2cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } } peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } }
I20260808 13:03:19.545848 3729 heartbeater.cc:503] Master 127.1.124.190:40289 was elected leader, sending a full tablet report...
I20260808 13:03:19.546454 3972 ts_tablet_manager.cc:1428] T 269b4cb903c14f889f57c4c42a740e1f P ef2eb4eb991b40f98a5ff14e6a2da2cd: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
W20260808 13:03:19.565640 3518 tablet.cc:2369] T 269b4cb903c14f889f57c4c42a740e1f P 5dea7daa69ce4fe2913c2e44e407469c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:03:19.604553 3730 tablet.cc:2369] T 269b4cb903c14f889f57c4c42a740e1f P ef2eb4eb991b40f98a5ff14e6a2da2cd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:03:19.630661 3938 tablet.cc:2369] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:03:19.670651 3979 raft_consensus.cc:482] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:03:19.671020 3979 raft_consensus.cc:504] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } } peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } }
I20260808 13:03:19.672894 3979 leader_election.cc:288] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5dea7daa69ce4fe2913c2e44e407469c (127.1.124.129:46485), ef2eb4eb991b40f98a5ff14e6a2da2cd (127.1.124.130:36257)
I20260808 13:03:19.680904 3968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:19.673344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:44839 (local address 127.1.124.130:36257)
0808 13:03:19.673570 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:19.673579 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:19.673604 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:03:19.673867 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:19.673877 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:19.673992 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:19.674107 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:19.674672 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.675932 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.678480 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.679747 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.679922 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:19.680013 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:19.680517 (+ 504us) server_negotiation.cc:299] Negotiation successful
0808 13:03:19.680646 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 13:03:19.680904 3965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:19.673364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:59947 (local address 127.1.124.129:46485)
0808 13:03:19.673558 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:19.673565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:19.674307 (+ 742us) server_negotiation.cc:407] Connection header received
0808 13:03:19.674455 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:19.674459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:19.674559 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:19.674686 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:19.675241 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.676336 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.678665 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.679900 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.680044 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:19.680169 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:19.680541 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 13:03:19.680679 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 13:03:19.681007 3770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:19.673186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:36257 (local address 127.1.124.131:44839)
0808 13:03:19.673494 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:19.673519 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:19.673657 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:19.674146 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:19.674151 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:19.674186 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:19.674516 (+ 330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:19.674527 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.676144 (+ 1617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.676149 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:19.678297 (+ 2148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:19.678312 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.679926 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.679931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:19.680287 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:19.680330 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 13:03:19.680437 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 13:03:19.680634 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":7}
I20260808 13:03:19.681125 3983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:19.673407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:46485 (local address 127.1.124.131:59947)
0808 13:03:19.674177 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:19.674194 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:19.674314 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:19.674701 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:19.674704 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:19.674725 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:19.675053 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:19.675060 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.676476 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.676481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:19.678496 (+ 2015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:19.678506 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.680187 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.680195 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:19.680364 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:19.680411 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 13:03:19.680499 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 13:03:19.680634 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"mutex_wait_us":3,"thread_start_us":450,"threads_started":1}
I20260808 13:03:19.682025 3660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "269b4cb903c14f889f57c4c42a740e1f" candidate_uuid: "b5eef982d5114c18a141bbbd703b44c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" is_pre_election: true
I20260808 13:03:19.682119 3438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "269b4cb903c14f889f57c4c42a740e1f" candidate_uuid: "b5eef982d5114c18a141bbbd703b44c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dea7daa69ce4fe2913c2e44e407469c" is_pre_election: true
I20260808 13:03:19.682559 3660 raft_consensus.cc:2454] T 269b4cb903c14f889f57c4c42a740e1f P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5eef982d5114c18a141bbbd703b44c4 in term 0.
I20260808 13:03:19.682669 3438 raft_consensus.cc:2454] T 269b4cb903c14f889f57c4c42a740e1f P 5dea7daa69ce4fe2913c2e44e407469c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5eef982d5114c18a141bbbd703b44c4 in term 0.
I20260808 13:03:19.683347 3756 leader_election.cc:302] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5dea7daa69ce4fe2913c2e44e407469c, b5eef982d5114c18a141bbbd703b44c4, ef2eb4eb991b40f98a5ff14e6a2da2cd; no voters:
I20260808 13:03:19.683863 3979 raft_consensus.cc:2781] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:03:19.684041 3979 raft_consensus.cc:482] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:03:19.684192 3979 raft_consensus.cc:3037] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:19.686709 3979 raft_consensus.cc:504] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } } peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } }
I20260808 13:03:19.687470 3979 leader_election.cc:288] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4 [CANDIDATE]: Term 1 election: Requested vote from peers 5dea7daa69ce4fe2913c2e44e407469c (127.1.124.129:46485), ef2eb4eb991b40f98a5ff14e6a2da2cd (127.1.124.130:36257)
I20260808 13:03:19.687927 3438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "269b4cb903c14f889f57c4c42a740e1f" candidate_uuid: "b5eef982d5114c18a141bbbd703b44c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dea7daa69ce4fe2913c2e44e407469c"
I20260808 13:03:19.688012 3660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "269b4cb903c14f889f57c4c42a740e1f" candidate_uuid: "b5eef982d5114c18a141bbbd703b44c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd"
I20260808 13:03:19.688206 3438 raft_consensus.cc:3037] T 269b4cb903c14f889f57c4c42a740e1f P 5dea7daa69ce4fe2913c2e44e407469c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:19.688256 3660 raft_consensus.cc:3037] T 269b4cb903c14f889f57c4c42a740e1f P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:19.691882 3660 raft_consensus.cc:2454] T 269b4cb903c14f889f57c4c42a740e1f P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5eef982d5114c18a141bbbd703b44c4 in term 1.
I20260808 13:03:19.691874 3438 raft_consensus.cc:2454] T 269b4cb903c14f889f57c4c42a740e1f P 5dea7daa69ce4fe2913c2e44e407469c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5eef982d5114c18a141bbbd703b44c4 in term 1.
I20260808 13:03:19.692364 3759 leader_election.cc:302] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5dea7daa69ce4fe2913c2e44e407469c, b5eef982d5114c18a141bbbd703b44c4, ef2eb4eb991b40f98a5ff14e6a2da2cd; no voters:
I20260808 13:03:19.692775 3979 raft_consensus.cc:2781] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:19.693236 3979 raft_consensus.cc:686] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4 [term 1 LEADER]: Becoming Leader. State: Replica: b5eef982d5114c18a141bbbd703b44c4, State: Running, Role: LEADER
I20260808 13:03:19.693791 3979 consensus_queue.cc:237] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } } peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } }
I20260808 13:03:19.698298 3209 catalog_manager.cc:5495] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5eef982d5114c18a141bbbd703b44c4 (127.1.124.131). New cstate: current_term: 1 leader_uuid: "b5eef982d5114c18a141bbbd703b44c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } health_report { overall_health: UNKNOWN } } }
I20260808 13:03:19.741214 3964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:19.735437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40708 (local address 127.1.124.131:39845)
0808 13:03:19.735642 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:19.735647 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:19.735915 (+ 268us) server_negotiation.cc:407] Connection header received
0808 13:03:19.736355 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:19.736359 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:19.736449 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:19.736578 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:03:19.736940 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.737724 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.739073 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.739475 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.739567 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:19.740056 (+ 489us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:03:19.740646 (+ 590us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:03:19.740781 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:19.740924 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 13:03:19.740988 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 13:03:19.750358 3660 raft_consensus.cc:1261] T 269b4cb903c14f889f57c4c42a740e1f P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 1 FOLLOWER]: Refusing update from remote peer b5eef982d5114c18a141bbbd703b44c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:03:19.751881 3438 raft_consensus.cc:1261] T 269b4cb903c14f889f57c4c42a740e1f P 5dea7daa69ce4fe2913c2e44e407469c [term 1 FOLLOWER]: Refusing update from remote peer b5eef982d5114c18a141bbbd703b44c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:03:19.751585 3984 consensus_queue.cc:1035] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:03:19.752799 3984 consensus_queue.cc:1035] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:03:19.767479 3989 mvcc.cc:205] Tried to move back new op lower bound from 7316251442164748288 to 7316251441949151232. Current Snapshot: MvccSnapshot[applied={T|T < 7316251442164748288}]
I20260808 13:03:19.790525 3208 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:03:19.791002 3208 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:03:19.795679 3208 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86: Sending DeleteTablet for 3 replicas of tablet 269b4cb903c14f889f57c4c42a740e1f
I20260808 13:03:19.797066 3408 tablet_service.cc:1514] Processing DeleteTablet for tablet 269b4cb903c14f889f57c4c42a740e1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:19 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:36668
I20260808 13:03:19.797497 3623 tablet_service.cc:1514] Processing DeleteTablet for tablet 269b4cb903c14f889f57c4c42a740e1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:19 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:57456
I20260808 13:03:19.797060 3828 tablet_service.cc:1514] Processing DeleteTablet for tablet 269b4cb903c14f889f57c4c42a740e1f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:19 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:40702
I20260808 13:03:19.805285 3208 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41484:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:03:19.806337 3208 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:03:19.807770 4004 tablet_replica.cc:330] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4: stopping tablet replica
I20260808 13:03:19.808013 3999 tablet_replica.cc:330] T 269b4cb903c14f889f57c4c42a740e1f P 5dea7daa69ce4fe2913c2e44e407469c: stopping tablet replica
I20260808 13:03:19.808431 4004 raft_consensus.cc:2229] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:03:19.808734 3999 raft_consensus.cc:2229] T 269b4cb903c14f889f57c4c42a740e1f P 5dea7daa69ce4fe2913c2e44e407469c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:03:19.809556 4004 raft_consensus.cc:2258] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:19.809741 3999 raft_consensus.cc:2258] T 269b4cb903c14f889f57c4c42a740e1f P 5dea7daa69ce4fe2913c2e44e407469c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:19.812222 4004 ts_tablet_manager.cc:1905] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:19.812206 3999 ts_tablet_manager.cc:1905] T 269b4cb903c14f889f57c4c42a740e1f P 5dea7daa69ce4fe2913c2e44e407469c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:19.814342 4003 tablet_replica.cc:330] T 269b4cb903c14f889f57c4c42a740e1f P ef2eb4eb991b40f98a5ff14e6a2da2cd: stopping tablet replica
I20260808 13:03:19.816507 3828 tablet_service.cc:1467] Processing CreateTablet for tablet 627bd1f220354cf4b88f0b3187d86b62 (DEFAULT_TABLE table=test-table [id=d6f13628e46a4330bbbad4e70ce4e9bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:19.816669 3408 tablet_service.cc:1467] Processing CreateTablet for tablet 627bd1f220354cf4b88f0b3187d86b62 (DEFAULT_TABLE table=test-table [id=d6f13628e46a4330bbbad4e70ce4e9bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:19.817004 3828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627bd1f220354cf4b88f0b3187d86b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:19.817140 3408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627bd1f220354cf4b88f0b3187d86b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:19.820605 4003 raft_consensus.cc:2229] T 269b4cb903c14f889f57c4c42a740e1f P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:03:19.821206 4003 raft_consensus.cc:2258] T 269b4cb903c14f889f57c4c42a740e1f P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:19.823627 4003 ts_tablet_manager.cc:1905] T 269b4cb903c14f889f57c4c42a740e1f P ef2eb4eb991b40f98a5ff14e6a2da2cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:19.824509 3971 tablet_bootstrap.cc:492] T 627bd1f220354cf4b88f0b3187d86b62 P 5dea7daa69ce4fe2913c2e44e407469c: Bootstrap starting.
I20260808 13:03:19.824715 3622 tablet_service.cc:1467] Processing CreateTablet for tablet 627bd1f220354cf4b88f0b3187d86b62 (DEFAULT_TABLE table=test-table [id=d6f13628e46a4330bbbad4e70ce4e9bf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:19.825225 3622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 627bd1f220354cf4b88f0b3187d86b62. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:19.826267 3971 tablet_bootstrap.cc:654] T 627bd1f220354cf4b88f0b3187d86b62 P 5dea7daa69ce4fe2913c2e44e407469c: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:19.827041 4004 ts_tablet_manager.cc:1918] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:03:19.827291 4004 log.cc:1196] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/269b4cb903c14f889f57c4c42a740e1f
I20260808 13:03:19.833422 4003 ts_tablet_manager.cc:1918] T 269b4cb903c14f889f57c4c42a740e1f P ef2eb4eb991b40f98a5ff14e6a2da2cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:03:19.834298 4004 ts_tablet_manager.cc:1939] T 269b4cb903c14f889f57c4c42a740e1f P b5eef982d5114c18a141bbbd703b44c4: Deleting consensus metadata
I20260808 13:03:19.835239 3970 tablet_bootstrap.cc:492] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4: Bootstrap starting.
I20260808 13:03:19.836594 3194 catalog_manager.cc:4841] TS b5eef982d5114c18a141bbbd703b44c4 (127.1.124.131:39845): tablet 269b4cb903c14f889f57c4c42a740e1f (table test-table [id=dc96a4a6ea2142f79fd4d85017980ca6]) successfully deleted
I20260808 13:03:19.837262 3970 tablet_bootstrap.cc:654] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:19.834457 3972 tablet_bootstrap.cc:492] T 627bd1f220354cf4b88f0b3187d86b62 P ef2eb4eb991b40f98a5ff14e6a2da2cd: Bootstrap starting.
I20260808 13:03:19.838649 3971 tablet_bootstrap.cc:492] T 627bd1f220354cf4b88f0b3187d86b62 P 5dea7daa69ce4fe2913c2e44e407469c: No bootstrap required, opened a new log
I20260808 13:03:19.838817 3971 ts_tablet_manager.cc:1397] T 627bd1f220354cf4b88f0b3187d86b62 P 5dea7daa69ce4fe2913c2e44e407469c: Time spent bootstrapping tablet: real 0.014s user 0.009s sys 0.004s
I20260808 13:03:19.838982 3972 tablet_bootstrap.cc:654] T 627bd1f220354cf4b88f0b3187d86b62 P ef2eb4eb991b40f98a5ff14e6a2da2cd: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:19.839856 3971 raft_consensus.cc:348] T 627bd1f220354cf4b88f0b3187d86b62 P 5dea7daa69ce4fe2913c2e44e407469c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } }
I20260808 13:03:19.840297 3971 raft_consensus.cc:374] T 627bd1f220354cf4b88f0b3187d86b62 P 5dea7daa69ce4fe2913c2e44e407469c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:19.840473 3971 raft_consensus.cc:729] T 627bd1f220354cf4b88f0b3187d86b62 P 5dea7daa69ce4fe2913c2e44e407469c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dea7daa69ce4fe2913c2e44e407469c, State: Initialized, Role: FOLLOWER
I20260808 13:03:19.840845 3971 consensus_queue.cc:260] T 627bd1f220354cf4b88f0b3187d86b62 P 5dea7daa69ce4fe2913c2e44e407469c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } }
I20260808 13:03:19.843278 4003 log.cc:1196] T 269b4cb903c14f889f57c4c42a740e1f P ef2eb4eb991b40f98a5ff14e6a2da2cd: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/269b4cb903c14f889f57c4c42a740e1f
I20260808 13:03:19.844130 4003 ts_tablet_manager.cc:1939] T 269b4cb903c14f889f57c4c42a740e1f P ef2eb4eb991b40f98a5ff14e6a2da2cd: Deleting consensus metadata
I20260808 13:03:19.844367 3971 ts_tablet_manager.cc:1428] T 627bd1f220354cf4b88f0b3187d86b62 P 5dea7daa69ce4fe2913c2e44e407469c: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260808 13:03:19.846150 3970 tablet_bootstrap.cc:492] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4: No bootstrap required, opened a new log
I20260808 13:03:19.846333 3970 ts_tablet_manager.cc:1397] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.005s
I20260808 13:03:19.847317 3970 raft_consensus.cc:348] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } }
I20260808 13:03:19.847591 3970 raft_consensus.cc:374] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:19.847718 3970 raft_consensus.cc:729] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5eef982d5114c18a141bbbd703b44c4, State: Initialized, Role: FOLLOWER
I20260808 13:03:19.848038 3970 consensus_queue.cc:260] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } }
I20260808 13:03:19.848627 3190 catalog_manager.cc:4841] TS ef2eb4eb991b40f98a5ff14e6a2da2cd (127.1.124.130:36257): tablet 269b4cb903c14f889f57c4c42a740e1f (table test-table [id=dc96a4a6ea2142f79fd4d85017980ca6]) successfully deleted
I20260808 13:03:19.849002 3970 ts_tablet_manager.cc:1428] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260808 13:03:19.852696 3999 ts_tablet_manager.cc:1918] T 269b4cb903c14f889f57c4c42a740e1f P 5dea7daa69ce4fe2913c2e44e407469c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:03:19.853027 3999 log.cc:1196] T 269b4cb903c14f889f57c4c42a740e1f P 5dea7daa69ce4fe2913c2e44e407469c: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/269b4cb903c14f889f57c4c42a740e1f
I20260808 13:03:19.853609 3999 ts_tablet_manager.cc:1939] T 269b4cb903c14f889f57c4c42a740e1f P 5dea7daa69ce4fe2913c2e44e407469c: Deleting consensus metadata
I20260808 13:03:19.855281 3193 catalog_manager.cc:4841] TS 5dea7daa69ce4fe2913c2e44e407469c (127.1.124.129:46485): tablet 269b4cb903c14f889f57c4c42a740e1f (table test-table [id=dc96a4a6ea2142f79fd4d85017980ca6]) successfully deleted
I20260808 13:03:19.857035 3972 tablet_bootstrap.cc:492] T 627bd1f220354cf4b88f0b3187d86b62 P ef2eb4eb991b40f98a5ff14e6a2da2cd: No bootstrap required, opened a new log
I20260808 13:03:19.857215 3972 ts_tablet_manager.cc:1397] T 627bd1f220354cf4b88f0b3187d86b62 P ef2eb4eb991b40f98a5ff14e6a2da2cd: Time spent bootstrapping tablet: real 0.023s user 0.006s sys 0.009s
I20260808 13:03:19.858119 3972 raft_consensus.cc:348] T 627bd1f220354cf4b88f0b3187d86b62 P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } }
I20260808 13:03:19.858389 3972 raft_consensus.cc:374] T 627bd1f220354cf4b88f0b3187d86b62 P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:19.858506 3972 raft_consensus.cc:729] T 627bd1f220354cf4b88f0b3187d86b62 P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2eb4eb991b40f98a5ff14e6a2da2cd, State: Initialized, Role: FOLLOWER
I20260808 13:03:19.858816 3972 consensus_queue.cc:260] T 627bd1f220354cf4b88f0b3187d86b62 P ef2eb4eb991b40f98a5ff14e6a2da2cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } }
I20260808 13:03:19.859540 3972 ts_tablet_manager.cc:1428] T 627bd1f220354cf4b88f0b3187d86b62 P ef2eb4eb991b40f98a5ff14e6a2da2cd: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 13:03:19.927690 3993 raft_consensus.cc:482] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:03:19.927973 3993 raft_consensus.cc:504] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } }
I20260808 13:03:19.928977 3993 leader_election.cc:288] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5dea7daa69ce4fe2913c2e44e407469c (127.1.124.129:46485), ef2eb4eb991b40f98a5ff14e6a2da2cd (127.1.124.130:36257)
I20260808 13:03:19.929466 3438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627bd1f220354cf4b88f0b3187d86b62" candidate_uuid: "b5eef982d5114c18a141bbbd703b44c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dea7daa69ce4fe2913c2e44e407469c" is_pre_election: true
I20260808 13:03:19.929610 3660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627bd1f220354cf4b88f0b3187d86b62" candidate_uuid: "b5eef982d5114c18a141bbbd703b44c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" is_pre_election: true
I20260808 13:03:19.929744 3438 raft_consensus.cc:2454] T 627bd1f220354cf4b88f0b3187d86b62 P 5dea7daa69ce4fe2913c2e44e407469c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5eef982d5114c18a141bbbd703b44c4 in term 0.
I20260808 13:03:19.929941 3660 raft_consensus.cc:2454] T 627bd1f220354cf4b88f0b3187d86b62 P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5eef982d5114c18a141bbbd703b44c4 in term 0.
I20260808 13:03:19.930191 3759 leader_election.cc:302] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dea7daa69ce4fe2913c2e44e407469c, b5eef982d5114c18a141bbbd703b44c4; no voters:
I20260808 13:03:19.930476 3993 raft_consensus.cc:2781] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:03:19.930632 3993 raft_consensus.cc:482] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:03:19.930749 3993 raft_consensus.cc:3037] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:19.933149 3993 raft_consensus.cc:504] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } }
I20260808 13:03:19.933719 3993 leader_election.cc:288] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4 [CANDIDATE]: Term 1 election: Requested vote from peers 5dea7daa69ce4fe2913c2e44e407469c (127.1.124.129:46485), ef2eb4eb991b40f98a5ff14e6a2da2cd (127.1.124.130:36257)
I20260808 13:03:19.934230 3438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627bd1f220354cf4b88f0b3187d86b62" candidate_uuid: "b5eef982d5114c18a141bbbd703b44c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dea7daa69ce4fe2913c2e44e407469c"
I20260808 13:03:19.934446 3438 raft_consensus.cc:3037] T 627bd1f220354cf4b88f0b3187d86b62 P 5dea7daa69ce4fe2913c2e44e407469c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:19.934479 3660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "627bd1f220354cf4b88f0b3187d86b62" candidate_uuid: "b5eef982d5114c18a141bbbd703b44c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd"
I20260808 13:03:19.934715 3660 raft_consensus.cc:3037] T 627bd1f220354cf4b88f0b3187d86b62 P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:19.936861 3438 raft_consensus.cc:2454] T 627bd1f220354cf4b88f0b3187d86b62 P 5dea7daa69ce4fe2913c2e44e407469c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5eef982d5114c18a141bbbd703b44c4 in term 1.
I20260808 13:03:19.937307 3759 leader_election.cc:302] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dea7daa69ce4fe2913c2e44e407469c, b5eef982d5114c18a141bbbd703b44c4; no voters:
I20260808 13:03:19.937598 3993 raft_consensus.cc:2781] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:19.937779 3993 raft_consensus.cc:686] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4 [term 1 LEADER]: Becoming Leader. State: Replica: b5eef982d5114c18a141bbbd703b44c4, State: Running, Role: LEADER
I20260808 13:03:19.937875 3660 raft_consensus.cc:2454] T 627bd1f220354cf4b88f0b3187d86b62 P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5eef982d5114c18a141bbbd703b44c4 in term 1.
I20260808 13:03:19.938139 3993 consensus_queue.cc:237] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } }
I20260808 13:03:19.941164 3208 catalog_manager.cc:5495] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5eef982d5114c18a141bbbd703b44c4 (127.1.124.131). New cstate: current_term: 1 leader_uuid: "b5eef982d5114c18a141bbbd703b44c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } health_report { overall_health: HEALTHY } } }
Aug 08 13:03:19 dist-test-slave-t3q3 krb5kdc[3145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194197, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:20.000696 4017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:19.982884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:40289 (local address 127.0.0.1:41496)
0808 13:03:19.983926 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:19.983949 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:19.984165 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:19.984747 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:19.984752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:19.985738 (+ 986us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:19.986072 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:19.986085 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.987307 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.987313 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:19.989044 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:19.989059 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.989892 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.989899 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:19.990092 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:19.990813 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:19.990844 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:19.993708 (+ 2864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:19.997767 (+ 4059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:19.997779 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:19.997785 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:19.998210 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:19.998622 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:19.998625 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:19.998627 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:19.998763 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:19.999432 (+ 669us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:19.999438 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:19.999626 (+ 188us) client_negotiation.cc:769] Sending connection context
0808 13:03:19.999825 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 13:03:20.000218 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"thread_start_us":617,"threads_started":1}
I20260808 13:03:20.000823 4016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:19.982872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41496 (local address 127.1.124.190:40289)
0808 13:03:19.984381 (+ 1509us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:19.984385 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:19.984423 (+ 38us) server_negotiation.cc:407] Connection header received
0808 13:03:19.984470 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:19.984473 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:19.984536 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:19.984628 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:19.986256 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.987164 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.989208 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:19.989678 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:19.989786 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:19.993945 (+ 4159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:19.993952 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:19.993957 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:19.993995 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:19.997609 (+ 3614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:19.998349 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:19.998355 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:19.998359 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:19.998504 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:19.998884 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:19.998889 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:19.998892 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:19.999260 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:19.999431 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:19.999936 (+ 505us) server_negotiation.cc:299] Negotiation successful
0808 13:03:20.000389 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":212,"server-negotiator.queue_time_us":1336,"thread_start_us":1166,"threads_started":1}
I20260808 13:03:20.006162 3208 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:41496:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 13:03:20.007328 3208 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:03:20.015350 3408 tablet_service.cc:1467] Processing CreateTablet for tablet 93024b4ddb234df389eec462f63022cb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a663243c46b748d5a65cd8a4aaf2749c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 13:03:20.016053 3408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93024b4ddb234df389eec462f63022cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:20.016649 3828 tablet_service.cc:1467] Processing CreateTablet for tablet 93024b4ddb234df389eec462f63022cb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a663243c46b748d5a65cd8a4aaf2749c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 13:03:20.017243 3828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93024b4ddb234df389eec462f63022cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:20.020709 3622 tablet_service.cc:1467] Processing CreateTablet for tablet 93024b4ddb234df389eec462f63022cb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=a663243c46b748d5a65cd8a4aaf2749c]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 13:03:20.021245 3622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93024b4ddb234df389eec462f63022cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:20.024350 3971 tablet_bootstrap.cc:492] T 93024b4ddb234df389eec462f63022cb P 5dea7daa69ce4fe2913c2e44e407469c: Bootstrap starting.
I20260808 13:03:20.025682 3970 tablet_bootstrap.cc:492] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4: Bootstrap starting.
I20260808 13:03:20.026218 3971 tablet_bootstrap.cc:654] T 93024b4ddb234df389eec462f63022cb P 5dea7daa69ce4fe2913c2e44e407469c: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:20.027778 3970 tablet_bootstrap.cc:654] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:20.028927 3972 tablet_bootstrap.cc:492] T 93024b4ddb234df389eec462f63022cb P ef2eb4eb991b40f98a5ff14e6a2da2cd: Bootstrap starting.
I20260808 13:03:20.030597 3972 tablet_bootstrap.cc:654] T 93024b4ddb234df389eec462f63022cb P ef2eb4eb991b40f98a5ff14e6a2da2cd: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:20.036454 3971 tablet_bootstrap.cc:492] T 93024b4ddb234df389eec462f63022cb P 5dea7daa69ce4fe2913c2e44e407469c: No bootstrap required, opened a new log
I20260808 13:03:20.036803 3971 ts_tablet_manager.cc:1397] T 93024b4ddb234df389eec462f63022cb P 5dea7daa69ce4fe2913c2e44e407469c: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.003s
I20260808 13:03:20.037449 3971 raft_consensus.cc:348] T 93024b4ddb234df389eec462f63022cb P 5dea7daa69ce4fe2913c2e44e407469c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } }
I20260808 13:03:20.037808 3971 raft_consensus.cc:374] T 93024b4ddb234df389eec462f63022cb P 5dea7daa69ce4fe2913c2e44e407469c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:20.037941 3971 raft_consensus.cc:729] T 93024b4ddb234df389eec462f63022cb P 5dea7daa69ce4fe2913c2e44e407469c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dea7daa69ce4fe2913c2e44e407469c, State: Initialized, Role: FOLLOWER
I20260808 13:03:20.038425 3970 tablet_bootstrap.cc:492] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4: No bootstrap required, opened a new log
I20260808 13:03:20.038290 3971 consensus_queue.cc:260] T 93024b4ddb234df389eec462f63022cb P 5dea7daa69ce4fe2913c2e44e407469c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } }
I20260808 13:03:20.039028 3972 tablet_bootstrap.cc:492] T 93024b4ddb234df389eec462f63022cb P ef2eb4eb991b40f98a5ff14e6a2da2cd: No bootstrap required, opened a new log
I20260808 13:03:20.039131 3971 ts_tablet_manager.cc:1428] T 93024b4ddb234df389eec462f63022cb P 5dea7daa69ce4fe2913c2e44e407469c: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 13:03:20.039234 3972 ts_tablet_manager.cc:1397] T 93024b4ddb234df389eec462f63022cb P ef2eb4eb991b40f98a5ff14e6a2da2cd: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.005s
I20260808 13:03:20.038975 3981 tablet_replica.cc:439] T 93024b4ddb234df389eec462f63022cb P 5dea7daa69ce4fe2913c2e44e407469c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } } }
I20260808 13:03:20.039492 3981 tablet_replica.cc:442] T 93024b4ddb234df389eec462f63022cb P 5dea7daa69ce4fe2913c2e44e407469c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:03:20.039963 3972 raft_consensus.cc:348] T 93024b4ddb234df389eec462f63022cb P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } }
I20260808 13:03:20.040246 3972 raft_consensus.cc:374] T 93024b4ddb234df389eec462f63022cb P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:20.040319 3972 raft_consensus.cc:729] T 93024b4ddb234df389eec462f63022cb P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef2eb4eb991b40f98a5ff14e6a2da2cd, State: Initialized, Role: FOLLOWER
I20260808 13:03:20.040822 3972 consensus_queue.cc:260] T 93024b4ddb234df389eec462f63022cb P ef2eb4eb991b40f98a5ff14e6a2da2cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } }
I20260808 13:03:20.041445 3972 ts_tablet_manager.cc:1428] T 93024b4ddb234df389eec462f63022cb P ef2eb4eb991b40f98a5ff14e6a2da2cd: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 13:03:20.041416 3982 tablet_replica.cc:439] T 93024b4ddb234df389eec462f63022cb P ef2eb4eb991b40f98a5ff14e6a2da2cd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } } }
I20260808 13:03:20.041818 3982 tablet_replica.cc:442] T 93024b4ddb234df389eec462f63022cb P ef2eb4eb991b40f98a5ff14e6a2da2cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:03:20.043411 3970 ts_tablet_manager.cc:1397] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.006s
I20260808 13:03:20.044524 3970 raft_consensus.cc:348] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } }
I20260808 13:03:20.044826 3970 raft_consensus.cc:374] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:20.044952 3970 raft_consensus.cc:729] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5eef982d5114c18a141bbbd703b44c4, State: Initialized, Role: FOLLOWER
I20260808 13:03:20.045269 3970 consensus_queue.cc:260] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } }
I20260808 13:03:20.045780 3993 tablet_replica.cc:439] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } } }
I20260808 13:03:20.045957 3970 ts_tablet_manager.cc:1428] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 13:03:20.046094 3993 tablet_replica.cc:442] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:03:20.098919 3993 raft_consensus.cc:482] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:03:20.099205 3993 raft_consensus.cc:504] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } }
I20260808 13:03:20.100113 3993 leader_election.cc:288] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5dea7daa69ce4fe2913c2e44e407469c (127.1.124.129:46485), ef2eb4eb991b40f98a5ff14e6a2da2cd (127.1.124.130:36257)
I20260808 13:03:20.100857 3438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93024b4ddb234df389eec462f63022cb" candidate_uuid: "b5eef982d5114c18a141bbbd703b44c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dea7daa69ce4fe2913c2e44e407469c" is_pre_election: true
I20260808 13:03:20.100893 3660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93024b4ddb234df389eec462f63022cb" candidate_uuid: "b5eef982d5114c18a141bbbd703b44c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" is_pre_election: true
I20260808 13:03:20.101177 3438 raft_consensus.cc:2454] T 93024b4ddb234df389eec462f63022cb P 5dea7daa69ce4fe2913c2e44e407469c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5eef982d5114c18a141bbbd703b44c4 in term 0.
I20260808 13:03:20.101181 3660 raft_consensus.cc:2454] T 93024b4ddb234df389eec462f63022cb P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5eef982d5114c18a141bbbd703b44c4 in term 0.
I20260808 13:03:20.101673 3756 leader_election.cc:302] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b5eef982d5114c18a141bbbd703b44c4, ef2eb4eb991b40f98a5ff14e6a2da2cd; no voters:
I20260808 13:03:20.102241 3993 raft_consensus.cc:2781] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:03:20.102349 3993 raft_consensus.cc:482] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:03:20.102463 3993 raft_consensus.cc:3037] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:20.106051 3993 raft_consensus.cc:504] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } }
I20260808 13:03:20.106870 3993 leader_election.cc:288] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4 [CANDIDATE]: Term 1 election: Requested vote from peers 5dea7daa69ce4fe2913c2e44e407469c (127.1.124.129:46485), ef2eb4eb991b40f98a5ff14e6a2da2cd (127.1.124.130:36257)
I20260808 13:03:20.107484 3438 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93024b4ddb234df389eec462f63022cb" candidate_uuid: "b5eef982d5114c18a141bbbd703b44c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5dea7daa69ce4fe2913c2e44e407469c"
I20260808 13:03:20.107506 3660 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93024b4ddb234df389eec462f63022cb" candidate_uuid: "b5eef982d5114c18a141bbbd703b44c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd"
I20260808 13:03:20.107746 3438 raft_consensus.cc:3037] T 93024b4ddb234df389eec462f63022cb P 5dea7daa69ce4fe2913c2e44e407469c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:20.107760 3660 raft_consensus.cc:3037] T 93024b4ddb234df389eec462f63022cb P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:20.110083 3438 raft_consensus.cc:2454] T 93024b4ddb234df389eec462f63022cb P 5dea7daa69ce4fe2913c2e44e407469c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5eef982d5114c18a141bbbd703b44c4 in term 1.
I20260808 13:03:20.110622 3759 leader_election.cc:302] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5dea7daa69ce4fe2913c2e44e407469c, b5eef982d5114c18a141bbbd703b44c4; no voters:
I20260808 13:03:20.110898 3993 raft_consensus.cc:2781] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:20.110994 3660 raft_consensus.cc:2454] T 93024b4ddb234df389eec462f63022cb P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5eef982d5114c18a141bbbd703b44c4 in term 1.
I20260808 13:03:20.111094 3993 raft_consensus.cc:686] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4 [term 1 LEADER]: Becoming Leader. State: Replica: b5eef982d5114c18a141bbbd703b44c4, State: Running, Role: LEADER
I20260808 13:03:20.111373 3993 consensus_queue.cc:237] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } }
I20260808 13:03:20.113417 3984 tablet_replica.cc:439] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4: TxnStatusTablet state changed. Reason: New leader b5eef982d5114c18a141bbbd703b44c4. Latest consensus state: current_term: 1 leader_uuid: "b5eef982d5114c18a141bbbd703b44c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } } }
I20260808 13:03:20.114025 3984 tablet_replica.cc:442] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4: This TxnStatusTablet replica's current role is: LEADER
I20260808 13:03:20.114893 4024 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 13:03:20.115239 4024 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 13:03:20.115515 3209 catalog_manager.cc:5495] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5eef982d5114c18a141bbbd703b44c4 (127.1.124.131). New cstate: current_term: 1 leader_uuid: "b5eef982d5114c18a141bbbd703b44c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } health_report { overall_health: HEALTHY } } }
I20260808 13:03:20.148157 3964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:20.141765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40716 (local address 127.1.124.131:39845)
0808 13:03:20.141960 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:20.141964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:20.142067 (+ 103us) server_negotiation.cc:407] Connection header received
0808 13:03:20.142313 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:20.142317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:20.142387 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:20.142470 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:03:20.143140 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:20.143976 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:20.146138 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:20.146565 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:20.146679 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:20.147278 (+ 599us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:03:20.147410 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:03:20.147489 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:20.147692 (+ 203us) server_negotiation.cc:299] Negotiation successful
0808 13:03:20.147759 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 13:03:20.148183 4017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:20.141641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:39845 (local address 127.0.0.1:40716)
0808 13:03:20.141925 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:20.141947 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:20.142122 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:20.142538 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:20.142543 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:20.142558 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:03:20.142961 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:20.142977 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:20.144143 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:20.144149 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:20.145963 (+ 1814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:20.145982 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:20.146753 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:20.146760 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:20.146975 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:20.147061 (+ 86us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:03:20.147530 (+ 469us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:03:20.147556 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 13:03:20.147667 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 13:03:20.147829 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 13:03:20.152412 3438 raft_consensus.cc:1261] T 93024b4ddb234df389eec462f63022cb P 5dea7daa69ce4fe2913c2e44e407469c [term 1 FOLLOWER]: Refusing update from remote peer b5eef982d5114c18a141bbbd703b44c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:03:20.152503 3660 raft_consensus.cc:1261] T 93024b4ddb234df389eec462f63022cb P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 1 FOLLOWER]: Refusing update from remote peer b5eef982d5114c18a141bbbd703b44c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:03:20.153249 3993 consensus_queue.cc:1035] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:03:20.153573 3984 consensus_queue.cc:1035] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:03:20.156255 3981 tablet_replica.cc:439] T 93024b4ddb234df389eec462f63022cb P 5dea7daa69ce4fe2913c2e44e407469c: TxnStatusTablet state changed. Reason: New leader b5eef982d5114c18a141bbbd703b44c4. Latest consensus state: current_term: 1 leader_uuid: "b5eef982d5114c18a141bbbd703b44c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } } }
I20260808 13:03:20.156572 3981 tablet_replica.cc:442] T 93024b4ddb234df389eec462f63022cb P 5dea7daa69ce4fe2913c2e44e407469c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:03:20.158803 3993 tablet_replica.cc:439] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b5eef982d5114c18a141bbbd703b44c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } } }
I20260808 13:03:20.159133 3993 tablet_replica.cc:442] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4: This TxnStatusTablet replica's current role is: LEADER
I20260808 13:03:20.160353 3987 mvcc.cc:205] Tried to move back new op lower bound from 7316251443817717760 to 7316251443659108352. Current Snapshot: MvccSnapshot[applied={T|T < 7316251443817717760 or (T in {7316251443817717760})}]
I20260808 13:03:20.161317 3981 tablet_replica.cc:439] T 93024b4ddb234df389eec462f63022cb P 5dea7daa69ce4fe2913c2e44e407469c: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b5eef982d5114c18a141bbbd703b44c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } } }
I20260808 13:03:20.161542 3981 tablet_replica.cc:442] T 93024b4ddb234df389eec462f63022cb P 5dea7daa69ce4fe2913c2e44e407469c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:03:20.162326 3988 mvcc.cc:205] Tried to move back new op lower bound from 7316251443817717760 to 7316251443659108352. Current Snapshot: MvccSnapshot[applied={T|T < 7316251443817717760 or (T in {7316251443817717760})}]
I20260808 13:03:20.168741 3982 tablet_replica.cc:439] T 93024b4ddb234df389eec462f63022cb P ef2eb4eb991b40f98a5ff14e6a2da2cd: TxnStatusTablet state changed. Reason: New leader b5eef982d5114c18a141bbbd703b44c4. Latest consensus state: current_term: 1 leader_uuid: "b5eef982d5114c18a141bbbd703b44c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } } }
I20260808 13:03:20.169131 3982 tablet_replica.cc:442] T 93024b4ddb234df389eec462f63022cb P ef2eb4eb991b40f98a5ff14e6a2da2cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:03:20.170385 3984 tablet_replica.cc:439] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b5eef982d5114c18a141bbbd703b44c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } } }
I20260808 13:03:20.170689 3984 tablet_replica.cc:442] T 93024b4ddb234df389eec462f63022cb P b5eef982d5114c18a141bbbd703b44c4: This TxnStatusTablet replica's current role is: LEADER
I20260808 13:03:20.172034 3982 tablet_replica.cc:439] T 93024b4ddb234df389eec462f63022cb P ef2eb4eb991b40f98a5ff14e6a2da2cd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b5eef982d5114c18a141bbbd703b44c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 } } peers { permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 } } peers { permanent_uuid: "b5eef982d5114c18a141bbbd703b44c4" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39845 } } }
I20260808 13:03:20.172278 3982 tablet_replica.cc:442] T 93024b4ddb234df389eec462f63022cb P ef2eb4eb991b40f98a5ff14e6a2da2cd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:03:20.187309 3964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:20.181262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:54485 (local address 127.1.124.131:39845)
0808 13:03:20.181677 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:20.181683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:20.182331 (+ 648us) server_negotiation.cc:407] Connection header received
0808 13:03:20.182559 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:20.182564 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:20.182634 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:20.182729 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:03:20.183130 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:20.183875 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:20.185365 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:20.185819 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:20.186009 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:20.186481 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:03:20.186604 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:03:20.186713 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:20.186855 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 13:03:20.186978 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263}
I20260808 13:03:20.187455 4032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:20.181284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:39845 (local address 127.1.124.131:54485)
0808 13:03:20.182222 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:20.182239 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:20.182412 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:20.182765 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:20.182769 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:20.182779 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:03:20.182985 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:20.182995 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:20.184023 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:20.184028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:20.185194 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:20.185207 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:20.186046 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:20.186052 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:20.186216 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:20.186270 (+ 54us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:03:20.186713 (+ 443us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:03:20.186737 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 13:03:20.186826 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 13:03:20.187038 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":820,"mutex_wait_us":56,"thread_start_us":745,"threads_started":1}
I20260808 13:03:20.199460 3660 raft_consensus.cc:1261] T 627bd1f220354cf4b88f0b3187d86b62 P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 1 FOLLOWER]: Refusing update from remote peer b5eef982d5114c18a141bbbd703b44c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:03:20.200183 3438 raft_consensus.cc:1261] T 627bd1f220354cf4b88f0b3187d86b62 P 5dea7daa69ce4fe2913c2e44e407469c [term 1 FOLLOWER]: Refusing update from remote peer b5eef982d5114c18a141bbbd703b44c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:03:20.200157 3984 consensus_queue.cc:1035] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef2eb4eb991b40f98a5ff14e6a2da2cd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:03:20.200747 3984 consensus_queue.cc:1035] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5dea7daa69ce4fe2913c2e44e407469c" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 46485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:03:20.263175 3208 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:03:20.263474 3208 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41484:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:03:20.266922 3208 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 804012c3878543fbaf20375887aa3e86: Sending DeleteTablet for 3 replicas of tablet 627bd1f220354cf4b88f0b3187d86b62
I20260808 13:03:20.267984 3622 tablet_service.cc:1514] Processing DeleteTablet for tablet 627bd1f220354cf4b88f0b3187d86b62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:20 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:57456
I20260808 13:03:20.267963 3408 tablet_service.cc:1514] Processing DeleteTablet for tablet 627bd1f220354cf4b88f0b3187d86b62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:20 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:36668
I20260808 13:03:20.268167 3828 tablet_service.cc:1514] Processing DeleteTablet for tablet 627bd1f220354cf4b88f0b3187d86b62 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:20 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:40702
I20260808 13:03:20.268309 3999 tablet_replica.cc:330] T 627bd1f220354cf4b88f0b3187d86b62 P 5dea7daa69ce4fe2913c2e44e407469c: stopping tablet replica
I20260808 13:03:20.268309 4003 tablet_replica.cc:330] T 627bd1f220354cf4b88f0b3187d86b62 P ef2eb4eb991b40f98a5ff14e6a2da2cd: stopping tablet replica
I20260808 13:03:20.268446 4004 tablet_replica.cc:330] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4: stopping tablet replica
I20260808 13:03:20.268524 4003 raft_consensus.cc:2229] T 627bd1f220354cf4b88f0b3187d86b62 P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:03:20.268524 3999 raft_consensus.cc:2229] T 627bd1f220354cf4b88f0b3187d86b62 P 5dea7daa69ce4fe2913c2e44e407469c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:03:20.268718 4003 raft_consensus.cc:2258] T 627bd1f220354cf4b88f0b3187d86b62 P ef2eb4eb991b40f98a5ff14e6a2da2cd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:20.268718 3999 raft_consensus.cc:2258] T 627bd1f220354cf4b88f0b3187d86b62 P 5dea7daa69ce4fe2913c2e44e407469c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:20.268703 4004 raft_consensus.cc:2229] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:03:20.269095 4004 raft_consensus.cc:2258] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:20.270313 4004 ts_tablet_manager.cc:1905] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:20.272785 3999 ts_tablet_manager.cc:1905] T 627bd1f220354cf4b88f0b3187d86b62 P 5dea7daa69ce4fe2913c2e44e407469c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:20.278510 4004 ts_tablet_manager.cc:1918] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:03:20.278702 4004 log.cc:1196] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/627bd1f220354cf4b88f0b3187d86b62
I20260808 13:03:20.279181 4004 ts_tablet_manager.cc:1939] T 627bd1f220354cf4b88f0b3187d86b62 P b5eef982d5114c18a141bbbd703b44c4: Deleting consensus metadata
I20260808 13:03:20.280211 3999 ts_tablet_manager.cc:1918] T 627bd1f220354cf4b88f0b3187d86b62 P 5dea7daa69ce4fe2913c2e44e407469c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:03:20.280398 3999 log.cc:1196] T 627bd1f220354cf4b88f0b3187d86b62 P 5dea7daa69ce4fe2913c2e44e407469c: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/627bd1f220354cf4b88f0b3187d86b62
I20260808 13:03:20.280886 3999 ts_tablet_manager.cc:1939] T 627bd1f220354cf4b88f0b3187d86b62 P 5dea7daa69ce4fe2913c2e44e407469c: Deleting consensus metadata
I20260808 13:03:20.281126 4003 ts_tablet_manager.cc:1905] T 627bd1f220354cf4b88f0b3187d86b62 P ef2eb4eb991b40f98a5ff14e6a2da2cd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:20.282644 3193 catalog_manager.cc:4841] TS 5dea7daa69ce4fe2913c2e44e407469c (127.1.124.129:46485): tablet 627bd1f220354cf4b88f0b3187d86b62 (table test-table [id=d6f13628e46a4330bbbad4e70ce4e9bf]) successfully deleted
Aug 08 13:03:20 dist-test-slave-t3q3 krb5kdc[3145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194199, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.124.129@KRBTEST.COM
I20260808 13:03:20.297840 4044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:20.271190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36672 (local address 127.1.124.129:46485)
0808 13:03:20.271738 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:20.271745 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:20.272626 (+ 881us) server_negotiation.cc:407] Connection header received
0808 13:03:20.273041 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:20.273048 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:20.273129 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:20.273221 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:20.275352 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:20.276650 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:20.278585 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:20.279400 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:20.279499 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:20.289143 (+ 9644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:20.289164 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:20.289186 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:20.289244 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:20.294092 (+ 4848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:20.295046 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:20.295058 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:20.295061 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:20.295249 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:20.295709 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:20.295714 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:20.295717 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:20.296244 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:20.296526 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:20.297152 (+ 626us) server_negotiation.cc:299] Negotiation successful
0808 13:03:20.297399 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":255,"threads_started":1}
W20260808 13:03:20.299759 3509 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36672
I20260808 13:03:20.303714 4003 ts_tablet_manager.cc:1918] T 627bd1f220354cf4b88f0b3187d86b62 P ef2eb4eb991b40f98a5ff14e6a2da2cd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:03:20.304159 4003 log.cc:1196] T 627bd1f220354cf4b88f0b3187d86b62 P ef2eb4eb991b40f98a5ff14e6a2da2cd: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/627bd1f220354cf4b88f0b3187d86b62
I20260808 13:03:20.304630 3194 catalog_manager.cc:4841] TS b5eef982d5114c18a141bbbd703b44c4 (127.1.124.131:39845): tablet 627bd1f220354cf4b88f0b3187d86b62 (table test-table [id=d6f13628e46a4330bbbad4e70ce4e9bf]) successfully deleted
I20260808 13:03:20.304714 4003 ts_tablet_manager.cc:1939] T 627bd1f220354cf4b88f0b3187d86b62 P ef2eb4eb991b40f98a5ff14e6a2da2cd: Deleting consensus metadata
I20260808 13:03:20.306078 3190 catalog_manager.cc:4841] TS ef2eb4eb991b40f98a5ff14e6a2da2cd (127.1.124.130:36257): tablet 627bd1f220354cf4b88f0b3187d86b62 (table test-table [id=d6f13628e46a4330bbbad4e70ce4e9bf]) successfully deleted
I20260808 13:03:20.334295 4016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:20.312330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41508 (local address 127.1.124.190:40289)
0808 13:03:20.312587 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:20.312597 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:20.313092 (+ 495us) server_negotiation.cc:407] Connection header received
0808 13:03:20.313410 (+ 318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:20.313418 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:20.313523 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:20.313637 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:20.315968 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:20.317359 (+ 1391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:20.319626 (+ 2267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:20.320475 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:20.320607 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:20.326765 (+ 6158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:20.326772 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:20.326776 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:20.326819 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:20.330758 (+ 3939us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:20.331443 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:20.331449 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:20.331453 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:20.331616 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:20.332120 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:20.332127 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:20.332133 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:20.332524 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:20.332773 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:20.333509 (+ 736us) server_negotiation.cc:299] Negotiation successful
0808 13:03:20.333744 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
W20260808 13:03:20.334962 3208 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41508
I20260808 13:03:20.337078 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 3313
I20260808 13:03:20.370424 4044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:20.337940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36678 (local address 127.1.124.129:46485)
0808 13:03:20.338108 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:20.338114 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:20.348649 (+ 10535us) server_negotiation.cc:407] Connection header received
0808 13:03:20.348983 (+ 334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:20.348989 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:20.349072 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:20.349146 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:20.352193 (+ 3047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:20.353544 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:20.360350 (+ 6806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:20.361216 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:20.361372 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:20.365098 (+ 3726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:20.365103 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:20.365106 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:20.365143 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:20.367745 (+ 2602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:20.368642 (+ 897us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:20.368646 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:20.368648 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:20.368770 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:20.369177 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:20.369180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:20.369182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:20.369429 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:20.369620 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:20.369990 (+ 370us) server_negotiation.cc:299] Negotiation successful
0808 13:03:20.370158 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 13:03:20.371106 3509 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:36678
W20260808 13:03:20.371692 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 3313: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:03:20.371889 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 3313
I20260808 13:03:20.408805 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 3521
Aug 08 13:03:20 dist-test-slave-t3q3 krb5kdc[3145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194199, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.124.130@KRBTEST.COM
I20260808 13:03:20.441220 4051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:20.409933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57464 (local address 127.1.124.130:36257)
0808 13:03:20.417818 (+ 7885us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:20.417825 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:20.417852 (+ 27us) server_negotiation.cc:407] Connection header received
0808 13:03:20.417931 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:20.417937 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:20.418027 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:20.419880 (+ 1853us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:20.420233 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:20.421392 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:20.423492 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:20.424267 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:20.424990 (+ 723us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:20.433095 (+ 8105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:20.433115 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:20.433135 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:20.433192 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:20.437008 (+ 3816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:20.437918 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:20.437927 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:20.437931 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:20.438142 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:20.438625 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:20.438630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:20.438633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:20.439105 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:20.440407 (+ 1302us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:20.440560 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 13:03:20.440802 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7708,"thread_start_us":7599,"threads_started":1}
W20260808 13:03:20.442251 3722 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57464
W20260808 13:03:20.442945 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 3521: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:03:20.443116 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 3521
I20260808 13:03:20.471535 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 3734
Aug 08 13:03:20 dist-test-slave-t3q3 krb5kdc[3145](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194199, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.124.131@KRBTEST.COM
I20260808 13:03:20.491160 3964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:20.472466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40718 (local address 127.1.124.131:39845)
0808 13:03:20.472677 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:20.472682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:20.472705 (+ 23us) server_negotiation.cc:407] Connection header received
0808 13:03:20.472917 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:20.472921 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:20.472997 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:20.473122 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:20.474850 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:20.475634 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:20.477576 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:20.478075 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:20.478176 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:20.483627 (+ 5451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:20.483652 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:20.483674 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:20.483737 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:20.487953 (+ 4216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:20.488765 (+ 812us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:20.488775 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:20.488779 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:20.488963 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:20.489366 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:20.489372 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:20.489375 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:20.489910 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:20.490247 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:20.490518 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 13:03:20.490772 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
W20260808 13:03:20.492002 3930 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40718
W20260808 13:03:20.492882 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 3734: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:03:20.493084 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 3734
I20260808 13:03:20.524914 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 3162
I20260808 13:03:20.526157 3265 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:20.643566 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 3162
2026-08-08T13:03:20Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (4960 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:03:20 dist-test-slave-t3q3 krb5kdc[4059](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:03:20 dist-test-slave-t3q3 krb5kdc[4059](info): set up 2 sockets
Aug 08 13:03:20 dist-test-slave-t3q3 krb5kdc[4059](info): commencing operation
krb5kdc: starting...
W20260808 13:03:21.983639 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.288s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:03:22 dist-test-slave-t3q3 krb5kdc[4059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:03:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:03:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:03:22.216602 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:37271
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:37271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:22.550030 4080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:22.550326 4080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:22.550421 4080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:22.557669 4080 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:03:22.557770 4080 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 13:03:22.557811 4080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:22.557874 4080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:22.557924 4080 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:03:22.557979 4080 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:03:22.567582 4080 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:37271
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:37271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:22.568879 4080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:22.570147 4080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:22.582301 4089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:22.582443 4086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:22.582417 4087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:22.583395 4080 server_base.cc:1034] running on GCE node
I20260808 13:03:22.584225 4080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:22.585578 4080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:22.586810 4080 hybrid_clock.cc:648] HybridClock initialized: now 1786194202586750 us; error 36 us; skew 500 ppm
I20260808 13:03:22.587193 4080 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:22 dist-test-slave-t3q3 krb5kdc[4059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194202, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:22.590320 4080 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:03:22.591854 4080 webserver.cc:460] Webserver started at http://127.1.124.190:40811/ using document root <none> and password file <none>
I20260808 13:03:22.592579 4080 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:22.592669 4080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:22.592947 4080 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:22.595860 4080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "67026e7bf983405485fc34735ed335a7"
format_stamp: "Formatted at 2026-08-08 13:03:22 on dist-test-slave-t3q3"
server_key: "262de8eac79cfe3684e76d3136a3631a"
server_key_iv: "859a325ba976bddbc231efc3c49ef137"
server_key_version: "encryptionkey@0"
I20260808 13:03:22.596658 4080 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "67026e7bf983405485fc34735ed335a7"
format_stamp: "Formatted at 2026-08-08 13:03:22 on dist-test-slave-t3q3"
server_key: "262de8eac79cfe3684e76d3136a3631a"
server_key_iv: "859a325ba976bddbc231efc3c49ef137"
server_key_version: "encryptionkey@0"
I20260808 13:03:22.602085 4080 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:03:22.605516 4101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:22.606828 4080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260808 13:03:22.606969 4080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "67026e7bf983405485fc34735ed335a7"
format_stamp: "Formatted at 2026-08-08 13:03:22 on dist-test-slave-t3q3"
server_key: "262de8eac79cfe3684e76d3136a3631a"
server_key_iv: "859a325ba976bddbc231efc3c49ef137"
server_key_version: "encryptionkey@0"
I20260808 13:03:22.607185 4080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:22.629645 4080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:22.634083 4080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:22.634299 4080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:22.656121 4080 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:37271
I20260808 13:03:22.656131 4191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:37271 every 8 connection(s)
I20260808 13:03:22.657629 4080 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:03:22.658921 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 4080
I20260808 13:03:22.659195 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:03:22.660118 1522 external_mini_cluster.cc:1428] Setting key 0c07c2c0edb6d41caecd471b1c894930
I20260808 13:03:22.663010 4192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 13:03:22 dist-test-slave-t3q3 krb5kdc[4059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194202, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:22.677913 4192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7: Bootstrap starting.
I20260808 13:03:22.681730 4192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:22.682896 4192 log.cc:824] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:22.685153 4196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:22.662381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51568 (local address 127.1.124.190:37271)
0808 13:03:22.663179 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:22.663191 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:22.663232 (+ 41us) server_negotiation.cc:407] Connection header received
0808 13:03:22.664941 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:22.664963 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:22.665334 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:22.665678 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:22.667690 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:22.669233 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:22.671239 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:22.671928 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:22.672141 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:22.677407 (+ 5266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:22.677416 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:22.677433 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:22.677472 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:22.680785 (+ 3313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:22.681571 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:22.681579 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:22.681586 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:22.681760 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:22.682190 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:22.682195 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:22.682201 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:22.682774 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:22.683141 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:22.683608 (+ 467us) server_negotiation.cc:299] Negotiation successful
0808 13:03:22.684110 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":560,"thread_start_us":365,"threads_started":1}
I20260808 13:03:22.686311 4192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7: No bootstrap required, opened a new log
I20260808 13:03:22.694384 4192 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67026e7bf983405485fc34735ed335a7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37271 } }
I20260808 13:03:22.694808 4192 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:22.694929 4192 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67026e7bf983405485fc34735ed335a7, State: Initialized, Role: FOLLOWER
I20260808 13:03:22.695607 4192 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67026e7bf983405485fc34735ed335a7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37271 } }
I20260808 13:03:22.695879 4192 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:03:22.696048 4192 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:03:22.696338 4192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:22.699079 4192 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67026e7bf983405485fc34735ed335a7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37271 } }
I20260808 13:03:22.699627 4192 leader_election.cc:302] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67026e7bf983405485fc34735ed335a7; no voters:
I20260808 13:03:22.700284 4192 leader_election.cc:288] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:03:22.700414 4199 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:22.701233 4199 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7 [term 1 LEADER]: Becoming Leader. State: Replica: 67026e7bf983405485fc34735ed335a7, State: Running, Role: LEADER
I20260808 13:03:22.701897 4199 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67026e7bf983405485fc34735ed335a7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37271 } }
I20260808 13:03:22.702718 4192 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:03:22.705017 4202 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67026e7bf983405485fc34735ed335a7. Latest consensus state: current_term: 1 leader_uuid: "67026e7bf983405485fc34735ed335a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67026e7bf983405485fc34735ed335a7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37271 } } }
I20260808 13:03:22.705466 4202 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7 [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:22.705047 4200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67026e7bf983405485fc34735ed335a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67026e7bf983405485fc34735ed335a7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37271 } } }
I20260808 13:03:22.705869 4200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7 [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:22.706501 4211 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:03:22.713006 4211 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:03:22.722093 4211 catalog_manager.cc:1347] Generated new cluster ID: e447e6d6a1e641f8809b99d1f37ad0a4
I20260808 13:03:22.722227 4211 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:03:22.741569 4211 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:03:22.743125 4211 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:03:22.754755 4211 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7: Generated new TSK 0
I20260808 13:03:22.755736 4211 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.124.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
I20260808 13:03:22.819324 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:37271
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:23.156509 4236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:23.156782 4236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:23.156953 4236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:23.164271 4236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:23.164382 4236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:23.164498 4236 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 13:03:23.164603 4236 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 13:03:23.164732 4236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:03:23.174639 4236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.124.190:37271
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:23.175915 4236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:23.177346 4236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:23.191093 4245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:23.191148 4243 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:23.191120 4242 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:23.191999 4236 server_base.cc:1034] running on GCE node
I20260808 13:03:23.192680 4236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:23.193725 4236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:23.194949 4236 hybrid_clock.cc:648] HybridClock initialized: now 1786194203194917 us; error 74 us; skew 500 ppm
I20260808 13:03:23.195283 4236 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:23 dist-test-slave-t3q3 krb5kdc[4059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194203, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:23.198554 4236 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:03:23.200127 4236 webserver.cc:460] Webserver started at http://127.1.124.129:45557/ using document root <none> and password file <none>
I20260808 13:03:23.200784 4236 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:23.200888 4236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:23.201175 4236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:23.204380 4236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505"
format_stamp: "Formatted at 2026-08-08 13:03:23 on dist-test-slave-t3q3"
server_key: "ec802b227623483a7a061397f1387cf2"
server_key_iv: "96f648653bba52f54020ca8182104768"
server_key_version: "encryptionkey@0"
I20260808 13:03:23.205106 4236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505"
format_stamp: "Formatted at 2026-08-08 13:03:23 on dist-test-slave-t3q3"
server_key: "ec802b227623483a7a061397f1387cf2"
server_key_iv: "96f648653bba52f54020ca8182104768"
server_key_version: "encryptionkey@0"
I20260808 13:03:23.210731 4236 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:03:23.214391 4259 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:23.215731 4236 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:03:23.215916 4236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505"
format_stamp: "Formatted at 2026-08-08 13:03:23 on dist-test-slave-t3q3"
server_key: "ec802b227623483a7a061397f1387cf2"
server_key_iv: "96f648653bba52f54020ca8182104768"
server_key_version: "encryptionkey@0"
I20260808 13:03:23.216188 4236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:23.231003 4236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:23.234863 4236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:23.235159 4236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:23.237773 4236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:23.237911 4236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:23.238054 4236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:23.238148 4236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 13:03:23 dist-test-slave-t3q3 krb5kdc[4059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194203, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:23.261845 4272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:23.238491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:43911 (local address 127.1.124.190:37271)
0808 13:03:23.239055 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:23.239064 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:23.240263 (+ 1199us) server_negotiation.cc:407] Connection header received
0808 13:03:23.242463 (+ 2200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:23.242468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:23.242557 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:23.242687 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:23.245093 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:23.246390 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:23.249001 (+ 2611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:23.249782 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:23.249952 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:23.255184 (+ 5232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:23.255189 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:23.255192 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:23.255223 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:23.258024 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:23.259068 (+ 1044us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:23.259074 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:23.259077 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:23.259271 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:23.259897 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:23.259904 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:23.259909 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:23.260288 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:23.260462 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:23.261224 (+ 762us) server_negotiation.cc:299] Negotiation successful
0808 13:03:23.261469 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":439,"thread_start_us":313,"threads_started":1}
I20260808 13:03:23.262815 4274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:23.239129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:37271 (local address 127.1.124.129:43911)
0808 13:03:23.240083 (+ 954us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:23.240139 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:23.242176 (+ 2037us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:23.242953 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:23.242967 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:23.243780 (+ 813us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:23.244791 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:23.244815 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:23.246572 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:23.246581 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:23.248835 (+ 2254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:23.248848 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:23.250057 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:23.250066 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:23.250424 (+ 358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:23.251256 (+ 832us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:23.251296 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:23.254641 (+ 3345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:23.258179 (+ 3538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:23.258188 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:23.258204 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:23.258927 (+ 723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:23.259409 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:23.259417 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:23.259424 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:23.259759 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:23.260534 (+ 775us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:23.260541 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:23.260861 (+ 320us) client_negotiation.cc:769] Sending connection context
0808 13:03:23.261135 (+ 274us) client_negotiation.cc:240] Negotiation successful
0808 13:03:23.261649 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":406,"threads_started":1}
I20260808 13:03:23.285208 4272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:23.269000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:59533 (local address 127.1.124.190:37271)
0808 13:03:23.269206 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:23.269214 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:23.269974 (+ 760us) server_negotiation.cc:407] Connection header received
0808 13:03:23.270129 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:23.270135 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:23.270205 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:23.270306 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:23.271511 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:23.272502 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:23.274569 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:23.275322 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:23.275471 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:23.277972 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:23.277978 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:23.277983 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:23.278024 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:23.281862 (+ 3838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:23.282958 (+ 1096us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:23.282965 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:23.282971 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:23.283191 (+ 220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:23.283634 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:23.283640 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:23.283643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:23.283921 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:23.284115 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:23.284595 (+ 480us) server_negotiation.cc:299] Negotiation successful
0808 13:03:23.284834 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 13:03:23.285360 4368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:23.269164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:37271 (local address 127.1.124.129:59533)
0808 13:03:23.269861 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:23.269880 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:23.269996 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:23.270335 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:23.270339 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:23.270914 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:23.271299 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:23.271319 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:23.272678 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:23.272684 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:23.274360 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:23.274402 (+ 42us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:23.275560 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:23.275569 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:23.275805 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:23.276566 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:23.276598 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:23.277761 (+ 1163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:23.282046 (+ 4285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:23.282054 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:23.282061 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:23.282832 (+ 771us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:23.283350 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:23.283358 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:23.283363 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:23.283503 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:23.284044 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:23.284049 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:23.284308 (+ 259us) client_negotiation.cc:769] Sending connection context
0808 13:03:23.284494 (+ 186us) client_negotiation.cc:240] Negotiation successful
0808 13:03:23.284873 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":378,"threads_started":1}
I20260808 13:03:23.290231 4236 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:37625
I20260808 13:03:23.290267 4439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:37625 every 8 connection(s)
I20260808 13:03:23.291435 4236 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:03:23.293509 4440 heartbeater.cc:347] Connected to a master server at 127.1.124.190:37271
I20260808 13:03:23.293745 4440 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:23.294394 4440 heartbeater.cc:511] Master 127.1.124.190:37271 requested a full tablet report, sending...
I20260808 13:03:23.296392 4124 ts_manager.cc:194] Registered new tserver with Master: fdde88d3d6dd4bf4a4bf7ca9ffbc4505 (127.1.124.129:37625)
I20260808 13:03:23.298202 4124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:43911
I20260808 13:03:23.301347 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 4236
I20260808 13:03:23.301573 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:03:23.302235 1522 external_mini_cluster.cc:1428] Setting key c6aa01085c096210502c39bddb1256d8
WARNING: no policy specified for kudu/127.1.124.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
I20260808 13:03:23.351917 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:37271
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:23.687988 4445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:23.688339 4445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:23.688511 4445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:23.695947 4445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:23.696137 4445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:23.696322 4445 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 13:03:23.696415 4445 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 13:03:23.696527 4445 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:03:23.706507 4445 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.124.190:37271
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:23.707988 4445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:23.709395 4445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:23.723316 4451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:23.723412 4457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:23.724172 4454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:23.725507 4445 server_base.cc:1034] running on GCE node
I20260808 13:03:23.726392 4445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:23.727787 4445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:23.729058 4445 hybrid_clock.cc:648] HybridClock initialized: now 1786194203728956 us; error 79 us; skew 500 ppm
I20260808 13:03:23.729518 4445 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:23 dist-test-slave-t3q3 krb5kdc[4059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194203, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:23.733863 4445 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:03:23.736248 4445 webserver.cc:460] Webserver started at http://127.1.124.130:36289/ using document root <none> and password file <none>
I20260808 13:03:23.737063 4445 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:23.737210 4445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:23.737553 4445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:23.740762 4445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "1019273dbea64a9ca91689ce685bc71a"
format_stamp: "Formatted at 2026-08-08 13:03:23 on dist-test-slave-t3q3"
server_key: "c6f4cfafd69b59bff1fc0f6cd1a25b26"
server_key_iv: "c01e3a3a3eb3fc49082de4a46cc99f50"
server_key_version: "encryptionkey@0"
I20260808 13:03:23.741602 4445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "1019273dbea64a9ca91689ce685bc71a"
format_stamp: "Formatted at 2026-08-08 13:03:23 on dist-test-slave-t3q3"
server_key: "c6f4cfafd69b59bff1fc0f6cd1a25b26"
server_key_iv: "c01e3a3a3eb3fc49082de4a46cc99f50"
server_key_version: "encryptionkey@0"
I20260808 13:03:23.747313 4445 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:03:23.751209 4471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:23.752707 4445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:03:23.752899 4445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "1019273dbea64a9ca91689ce685bc71a"
format_stamp: "Formatted at 2026-08-08 13:03:23 on dist-test-slave-t3q3"
server_key: "c6f4cfafd69b59bff1fc0f6cd1a25b26"
server_key_iv: "c01e3a3a3eb3fc49082de4a46cc99f50"
server_key_version: "encryptionkey@0"
I20260808 13:03:23.753136 4445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:23.790525 4445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:23.795670 4445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:23.796131 4445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:23.799145 4445 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:23.799306 4445 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:23.799453 4445 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:23.799558 4445 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 13:03:23 dist-test-slave-t3q3 krb5kdc[4059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194203, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:23.826277 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:23.800138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:56053 (local address 127.1.124.190:37271)
0808 13:03:23.802578 (+ 2440us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:23.802586 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:23.802611 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:03:23.804548 (+ 1937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:23.804557 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:23.804663 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:23.804826 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:23.807334 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:23.808950 (+ 1616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:23.811661 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:23.812191 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:23.812354 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:23.817403 (+ 5049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:23.817413 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:23.817421 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:23.817480 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:23.822068 (+ 4588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:23.823269 (+ 1201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:23.823273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:23.823276 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:23.823399 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:23.823898 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:23.823905 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:23.823908 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:23.824357 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:23.824512 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:23.825416 (+ 904us) server_negotiation.cc:299] Negotiation successful
0808 13:03:23.825695 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2292,"thread_start_us":2173,"threads_started":1}
I20260808 13:03:23.827358 4484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:23.800731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:37271 (local address 127.1.124.130:56053)
0808 13:03:23.801705 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:23.801761 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:23.804158 (+ 2397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:23.805180 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:23.805191 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:23.806055 (+ 864us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:23.807025 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:23.807058 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:23.809136 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:23.809144 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:23.811459 (+ 2315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:23.811483 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:23.812499 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:23.812513 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:23.812887 (+ 374us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:23.813711 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:23.813762 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:23.817126 (+ 3364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:23.822311 (+ 5185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:23.822326 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:23.822355 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:23.823056 (+ 701us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:23.823535 (+ 479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:23.823540 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:23.823544 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:23.823768 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:23.824598 (+ 830us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:23.824604 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:23.825056 (+ 452us) client_negotiation.cc:769] Sending connection context
0808 13:03:23.825343 (+ 287us) client_negotiation.cc:240] Negotiation successful
0808 13:03:23.826130 (+ 787us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":680,"thread_start_us":428,"threads_started":1}
I20260808 13:03:23.850129 4584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:23.834085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:37271 (local address 127.1.124.130:33441)
0808 13:03:23.834718 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:23.834734 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:23.834918 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:23.835321 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:23.835325 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:23.836119 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:23.836653 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:23.836674 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:23.837721 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:23.837726 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:23.838798 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:23.838811 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:23.839620 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:23.839626 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:23.839812 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:23.840537 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:23.840579 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:23.841861 (+ 1282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:23.846769 (+ 4908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:23.846787 (+ 18us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:23.846792 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:23.847419 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:23.847931 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:23.847939 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:23.847946 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:23.848144 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:23.848891 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:23.848896 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:23.849186 (+ 290us) client_negotiation.cc:769] Sending connection context
0808 13:03:23.849373 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 13:03:23.849702 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":347,"threads_started":1}
I20260808 13:03:23.850909 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:23.834006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:33441 (local address 127.1.124.190:37271)
0808 13:03:23.834172 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:23.834177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:23.834833 (+ 656us) server_negotiation.cc:407] Connection header received
0808 13:03:23.835110 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:23.835115 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:23.835181 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:23.835289 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:23.836836 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:23.837575 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:23.839016 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:23.839418 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:23.839554 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:23.842134 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:23.842145 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:23.842153 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:23.842209 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:23.846563 (+ 4354us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:23.847572 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:23.847581 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:23.847587 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:23.847786 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:23.848267 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:23.848273 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:23.848276 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:23.848778 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:23.848932 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:23.850496 (+ 1564us) server_negotiation.cc:299] Negotiation successful
0808 13:03:23.850700 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 13:03:23.856784 4445 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:41619
I20260808 13:03:23.856873 4647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:41619 every 8 connection(s)
I20260808 13:03:23.858153 4445 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:03:23.860491 4648 heartbeater.cc:347] Connected to a master server at 127.1.124.190:37271
I20260808 13:03:23.860749 4648 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:23.861447 4648 heartbeater.cc:511] Master 127.1.124.190:37271 requested a full tablet report, sending...
I20260808 13:03:23.862849 4124 ts_manager.cc:194] Registered new tserver with Master: 1019273dbea64a9ca91689ce685bc71a (127.1.124.130:41619)
I20260808 13:03:23.863692 4124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:56053
I20260808 13:03:23.866703 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 4445
I20260808 13:03:23.866925 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:03:23.867373 1522 external_mini_cluster.cc:1428] Setting key ecdee585fcb17395dbd62546fb88710c
WARNING: no policy specified for kudu/127.1.124.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
I20260808 13:03:23.915889 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:37271
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:42517
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:24.253141 4653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:24.253417 4653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:24.253611 4653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:24.260974 4653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:24.261080 4653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:24.261267 4653 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 13:03:24.261349 4653 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 13:03:24.261483 4653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:03:24.271142 4653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:42517
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.124.190:37271
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:24.272464 4653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:24.273860 4653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:24.287611 4661 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:24.287606 4659 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:24.287901 4665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:24.288520 4653 server_base.cc:1034] running on GCE node
I20260808 13:03:24.289093 4653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:24.290047 4653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:24.291286 4653 hybrid_clock.cc:648] HybridClock initialized: now 1786194204291226 us; error 49 us; skew 500 ppm
I20260808 13:03:24.291644 4653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:24 dist-test-slave-t3q3 krb5kdc[4059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194204, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:24.295090 4653 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:03:24.296566 4653 webserver.cc:460] Webserver started at http://127.1.124.131:38343/ using document root <none> and password file <none>
I20260808 13:03:24.297240 4653 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:24.297372 4653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:24.297696 4653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:24.300614 4653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "c182a7f66f6d49119a563266572c43b8"
format_stamp: "Formatted at 2026-08-08 13:03:24 on dist-test-slave-t3q3"
server_key: "a1b9bc87b6bfdcbe23e8653076655b0d"
server_key_iv: "aa57b18fe7f6a75a0825a49d4b8fd571"
server_key_version: "encryptionkey@0"
I20260808 13:03:24.301184 4440 heartbeater.cc:503] Master 127.1.124.190:37271 was elected leader, sending a full tablet report...
I20260808 13:03:24.301426 4653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "c182a7f66f6d49119a563266572c43b8"
format_stamp: "Formatted at 2026-08-08 13:03:24 on dist-test-slave-t3q3"
server_key: "a1b9bc87b6bfdcbe23e8653076655b0d"
server_key_iv: "aa57b18fe7f6a75a0825a49d4b8fd571"
server_key_version: "encryptionkey@0"
I20260808 13:03:24.307727 4653 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.004s
I20260808 13:03:24.311515 4675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:24.312980 4653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 13:03:24.313161 4653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "c182a7f66f6d49119a563266572c43b8"
format_stamp: "Formatted at 2026-08-08 13:03:24 on dist-test-slave-t3q3"
server_key: "a1b9bc87b6bfdcbe23e8653076655b0d"
server_key_iv: "aa57b18fe7f6a75a0825a49d4b8fd571"
server_key_version: "encryptionkey@0"
I20260808 13:03:24.313380 4653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:24.342717 4653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:24.346987 4653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:24.347280 4653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:24.350152 4653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:24.350271 4653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:24.350443 4653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:24.350551 4653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 13:03:24 dist-test-slave-t3q3 krb5kdc[4059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194204, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:24.375000 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.350881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:53537 (local address 127.1.124.190:37271)
0808 13:03:24.351104 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:24.351111 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:24.352380 (+ 1269us) server_negotiation.cc:407] Connection header received
0808 13:03:24.354563 (+ 2183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:24.354578 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:24.354667 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:24.354782 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:24.357234 (+ 2452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.358429 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.361192 (+ 2763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.361826 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.361991 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.366932 (+ 4941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:24.366937 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:24.366940 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:24.366971 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:24.370274 (+ 3303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:24.371440 (+ 1166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:24.371446 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:24.371449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:24.371637 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:24.372446 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:24.372453 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:24.372459 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:24.372919 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:24.373168 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:24.374150 (+ 982us) server_negotiation.cc:299] Negotiation successful
0808 13:03:24.374467 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 13:03:24.375931 4686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.351364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:37271 (local address 127.1.124.131:53537)
0808 13:03:24.352219 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:24.352264 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:24.354235 (+ 1971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:24.355098 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:24.355113 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:24.355885 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:24.356941 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:24.356997 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.358668 (+ 1671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.358675 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:24.361022 (+ 2347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:24.361048 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.362332 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.362338 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:24.362640 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.363496 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:24.363535 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:24.366691 (+ 3156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:24.370474 (+ 3783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:24.370483 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:24.370502 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:24.371261 (+ 759us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:24.371815 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:24.371823 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:24.371833 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:24.372269 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:24.373114 (+ 845us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:24.373127 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:24.373598 (+ 471us) client_negotiation.cc:769] Sending connection context
0808 13:03:24.373972 (+ 374us) client_negotiation.cc:240] Negotiation successful
0808 13:03:24.374413 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":352,"threads_started":1}
I20260808 13:03:24.398942 4653 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:34681
I20260808 13:03:24.399050 4853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:34681 every 8 connection(s)
I20260808 13:03:24.399950 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.383791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:38867 (local address 127.1.124.190:37271)
0808 13:03:24.383951 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:24.383956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:24.384308 (+ 352us) server_negotiation.cc:407] Connection header received
0808 13:03:24.384663 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:24.384669 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:24.384752 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:24.384875 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:24.386336 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.387315 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.389422 (+ 2107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.389893 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.390045 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.392864 (+ 2819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:24.392873 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:24.392880 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:24.392924 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:24.396616 (+ 3692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:24.397534 (+ 918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:24.397542 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:24.397554 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:24.397753 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:24.398292 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:24.398299 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:24.398303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:24.398718 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:24.398891 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:24.399529 (+ 638us) server_negotiation.cc:299] Negotiation successful
0808 13:03:24.399702 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 13:03:24.400264 4793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.383322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:37271 (local address 127.1.124.131:38867)
0808 13:03:24.384175 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:24.384202 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:24.384463 (+ 261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:24.384905 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:24.384909 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:24.385662 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:24.386190 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:24.386203 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.387484 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.387516 (+ 32us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:24.389270 (+ 1754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:24.389283 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.390147 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.390154 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:24.390289 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.391022 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:24.391065 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:24.392627 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:24.396759 (+ 4132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:24.396764 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:24.396768 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:24.397307 (+ 539us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:24.397907 (+ 600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:24.397915 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:24.397922 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:24.398132 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:24.398869 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:24.398874 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:24.399243 (+ 369us) client_negotiation.cc:769] Sending connection context
0808 13:03:24.399418 (+ 175us) client_negotiation.cc:240] Negotiation successful
0808 13:03:24.399780 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":633,"threads_started":1}
I20260808 13:03:24.400921 4653 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:03:24.404520 4854 heartbeater.cc:347] Connected to a master server at 127.1.124.190:37271
I20260808 13:03:24.404945 4854 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:24.405591 4854 heartbeater.cc:511] Master 127.1.124.190:37271 requested a full tablet report, sending...
I20260808 13:03:24.407132 4124 ts_manager.cc:194] Registered new tserver with Master: c182a7f66f6d49119a563266572c43b8 (127.1.124.131:34681)
I20260808 13:03:24.408047 4124 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:53537
I20260808 13:03:24.408317 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 4653
I20260808 13:03:24.408536 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:03:24.409116 1522 external_mini_cluster.cc:1428] Setting key 8b9396ad9c95f69409c24f1a5c4f7127
I20260808 13:03:24.412237 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 13:03:24 dist-test-slave-t3q3 krb5kdc[4059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 13:03:24 dist-test-slave-t3q3 krb5kdc[4059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194204, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:24.465675 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.451985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51584 (local address 127.1.124.190:37271)
0808 13:03:24.452189 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:24.452194 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:24.452683 (+ 489us) server_negotiation.cc:407] Connection header received
0808 13:03:24.452840 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:24.452844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:24.452908 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:24.453012 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:24.454498 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.455228 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.456534 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.457005 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.457141 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.460944 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:24.460948 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:24.460951 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:24.460981 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:24.463379 (+ 2398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:24.463962 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:24.463965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:24.463967 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:24.464139 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:24.464470 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:24.464473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:24.464475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:24.464753 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:24.464980 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:24.465254 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 13:03:24.465418 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 13:03:24.469722 4124 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51584:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:03:24.476680 4124 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:03:24.510665 4877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.489944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40020 (local address 127.1.124.129:37625)
0808 13:03:24.500843 (+ 10899us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:24.500854 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:24.500890 (+ 36us) server_negotiation.cc:407] Connection header received
0808 13:03:24.501018 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:24.501029 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:24.501323 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:24.501543 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:24.502499 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.504556 (+ 2057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.507846 (+ 3290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.509469 (+ 1623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.509613 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.509833 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:24.510154 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 13:03:24.510338 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10723,"thread_start_us":10577,"threads_started":1}
I20260808 13:03:24.510787 4876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.489752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:37625 (local address 127.0.0.1:40020)
0808 13:03:24.490593 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:24.490643 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:24.490821 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:24.501606 (+ 10785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:24.501617 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:24.501675 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:24.502275 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:24.502293 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.504721 (+ 2428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.504728 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:24.507683 (+ 2955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:24.507697 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.509680 (+ 1983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.509687 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:24.509891 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.509943 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 13:03:24.510092 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 13:03:24.510303 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"mutex_wait_us":364,"thread_start_us":11959,"threads_started":1}
I20260808 13:03:24.510911 4875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.489824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:41619 (local address 127.0.0.1:37938)
0808 13:03:24.501155 (+ 11331us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:24.501174 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:24.501309 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:24.502201 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:24.502206 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:24.502242 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:24.502709 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:24.502722 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.504741 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.504749 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:24.507824 (+ 3075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:24.507838 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.509990 (+ 2152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.509995 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:24.510196 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.510248 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 13:03:24.510337 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 13:03:24.510474 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11212,"mutex_wait_us":82,"thread_start_us":721,"threads_started":1}
I20260808 13:03:24.511915 4878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.490390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37938 (local address 127.1.124.130:41619)
0808 13:03:24.491345 (+ 955us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:24.491353 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:24.501451 (+ 10098us) server_negotiation.cc:407] Connection header received
0808 13:03:24.501558 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:24.501576 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:24.501798 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:24.501990 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:24.502993 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.504600 (+ 1607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.508017 (+ 3417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.509469 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.509613 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.509837 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:24.511390 (+ 1553us) server_negotiation.cc:299] Negotiation successful
0808 13:03:24.511598 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":806,"thread_start_us":1612,"threads_started":1}
I20260808 13:03:24.515228 4335 tablet_service.cc:1467] Processing CreateTablet for tablet be7ec46508e24730b1b6abaf83a81a67 (DEFAULT_TABLE table=test-table [id=d27f86e2f0b54f65b0d687c2888ab668]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:24.515228 4538 tablet_service.cc:1467] Processing CreateTablet for tablet be7ec46508e24730b1b6abaf83a81a67 (DEFAULT_TABLE table=test-table [id=d27f86e2f0b54f65b0d687c2888ab668]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:24.515919 4879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.491344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:34681 (local address 127.0.0.1:37056)
0808 13:03:24.507290 (+ 15946us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:24.507305 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:24.507457 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:24.508238 (+ 781us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:24.508242 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:24.508268 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:24.508646 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:24.508655 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.511615 (+ 2960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.511620 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:24.513468 (+ 1848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:24.513481 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.515196 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.515202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:24.515336 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.515359 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 13:03:24.515468 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 13:03:24.515597 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15824,"thread_start_us":15629,"threads_started":1}
I20260808 13:03:24.516001 4880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.491819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37056 (local address 127.1.124.131:34681)
0808 13:03:24.505479 (+ 13660us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:24.505490 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:24.507425 (+ 1935us) server_negotiation.cc:407] Connection header received
0808 13:03:24.507705 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:24.507712 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:24.507983 (+ 271us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:24.508263 (+ 280us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:24.508845 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.511468 (+ 2623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.513636 (+ 2168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.514640 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.514785 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.515043 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:24.515520 (+ 477us) server_negotiation.cc:299] Negotiation successful
0808 13:03:24.515684 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13417,"thread_start_us":13667,"threads_started":1}
I20260808 13:03:24.516834 4538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be7ec46508e24730b1b6abaf83a81a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:24.517534 4335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be7ec46508e24730b1b6abaf83a81a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:24.519366 4747 tablet_service.cc:1467] Processing CreateTablet for tablet be7ec46508e24730b1b6abaf83a81a67 (DEFAULT_TABLE table=test-table [id=d27f86e2f0b54f65b0d687c2888ab668]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:24.520644 4747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be7ec46508e24730b1b6abaf83a81a67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:24.531769 4881 tablet_bootstrap.cc:492] T be7ec46508e24730b1b6abaf83a81a67 P c182a7f66f6d49119a563266572c43b8: Bootstrap starting.
I20260808 13:03:24.535846 4881 tablet_bootstrap.cc:654] T be7ec46508e24730b1b6abaf83a81a67 P c182a7f66f6d49119a563266572c43b8: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:24.536608 4884 tablet_bootstrap.cc:492] T be7ec46508e24730b1b6abaf83a81a67 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Bootstrap starting.
I20260808 13:03:24.536916 4882 tablet_bootstrap.cc:492] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a: Bootstrap starting.
I20260808 13:03:24.537354 4881 log.cc:824] T be7ec46508e24730b1b6abaf83a81a67 P c182a7f66f6d49119a563266572c43b8: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:24.540247 4884 tablet_bootstrap.cc:654] T be7ec46508e24730b1b6abaf83a81a67 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:24.540613 4882 tablet_bootstrap.cc:654] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:24.541117 4881 tablet_bootstrap.cc:492] T be7ec46508e24730b1b6abaf83a81a67 P c182a7f66f6d49119a563266572c43b8: No bootstrap required, opened a new log
I20260808 13:03:24.541399 4881 ts_tablet_manager.cc:1397] T be7ec46508e24730b1b6abaf83a81a67 P c182a7f66f6d49119a563266572c43b8: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260808 13:03:24.541498 4884 log.cc:824] T be7ec46508e24730b1b6abaf83a81a67 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:24.541844 4882 log.cc:824] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:24.544657 4884 tablet_bootstrap.cc:492] T be7ec46508e24730b1b6abaf83a81a67 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: No bootstrap required, opened a new log
I20260808 13:03:24.544935 4884 ts_tablet_manager.cc:1397] T be7ec46508e24730b1b6abaf83a81a67 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260808 13:03:24.549969 4881 raft_consensus.cc:348] T be7ec46508e24730b1b6abaf83a81a67 P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:24.550449 4881 raft_consensus.cc:374] T be7ec46508e24730b1b6abaf83a81a67 P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:24.550559 4882 tablet_bootstrap.cc:492] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a: No bootstrap required, opened a new log
I20260808 13:03:24.550567 4881 raft_consensus.cc:729] T be7ec46508e24730b1b6abaf83a81a67 P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c182a7f66f6d49119a563266572c43b8, State: Initialized, Role: FOLLOWER
I20260808 13:03:24.550864 4882 ts_tablet_manager.cc:1397] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.000s
I20260808 13:03:24.551296 4881 consensus_queue.cc:260] T be7ec46508e24730b1b6abaf83a81a67 P c182a7f66f6d49119a563266572c43b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:24.553665 4884 raft_consensus.cc:348] T be7ec46508e24730b1b6abaf83a81a67 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:24.554419 4884 raft_consensus.cc:374] T be7ec46508e24730b1b6abaf83a81a67 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:24.554545 4884 raft_consensus.cc:729] T be7ec46508e24730b1b6abaf83a81a67 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdde88d3d6dd4bf4a4bf7ca9ffbc4505, State: Initialized, Role: FOLLOWER
I20260808 13:03:24.555327 4884 consensus_queue.cc:260] T be7ec46508e24730b1b6abaf83a81a67 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:24.557264 4884 ts_tablet_manager.cc:1428] T be7ec46508e24730b1b6abaf83a81a67 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Time spent starting tablet: real 0.012s user 0.010s sys 0.001s
I20260808 13:03:24.560415 4854 heartbeater.cc:503] Master 127.1.124.190:37271 was elected leader, sending a full tablet report...
I20260808 13:03:24.561440 4881 ts_tablet_manager.cc:1428] T be7ec46508e24730b1b6abaf83a81a67 P c182a7f66f6d49119a563266572c43b8: Time spent starting tablet: real 0.020s user 0.010s sys 0.009s
I20260808 13:03:24.564162 4882 raft_consensus.cc:348] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:24.564689 4882 raft_consensus.cc:374] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:24.564811 4882 raft_consensus.cc:729] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1019273dbea64a9ca91689ce685bc71a, State: Initialized, Role: FOLLOWER
I20260808 13:03:24.565478 4882 consensus_queue.cc:260] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:24.566773 4648 heartbeater.cc:503] Master 127.1.124.190:37271 was elected leader, sending a full tablet report...
I20260808 13:03:24.567296 4882 ts_tablet_manager.cc:1428] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a: Time spent starting tablet: real 0.016s user 0.017s sys 0.000s
I20260808 13:03:24.578874 4892 raft_consensus.cc:482] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:03:24.579159 4892 raft_consensus.cc:504] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:24.580932 4892 leader_election.cc:288] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fdde88d3d6dd4bf4a4bf7ca9ffbc4505 (127.1.124.129:37625), c182a7f66f6d49119a563266572c43b8 (127.1.124.131:34681)
I20260808 13:03:24.588872 4877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.581524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:55645 (local address 127.1.124.129:37625)
0808 13:03:24.581765 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:24.581774 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:24.582481 (+ 707us) server_negotiation.cc:407] Connection header received
0808 13:03:24.582742 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:24.582752 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:24.582838 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:24.582927 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:24.583995 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.585015 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.586965 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.587864 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.588096 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.588174 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:24.588480 (+ 306us) server_negotiation.cc:299] Negotiation successful
0808 13:03:24.588609 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 13:03:24.588999 4880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.581526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:38055 (local address 127.1.124.131:34681)
0808 13:03:24.581893 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:24.581902 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:24.582798 (+ 896us) server_negotiation.cc:407] Connection header received
0808 13:03:24.582996 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:24.582999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:24.583059 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:24.583157 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:24.583949 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.584928 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.587010 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.588015 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.588270 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.588345 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:24.588614 (+ 269us) server_negotiation.cc:299] Negotiation successful
0808 13:03:24.588741 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157}
I20260808 13:03:24.589087 4894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.581357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:37625 (local address 127.1.124.130:55645)
0808 13:03:24.582286 (+ 929us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:24.582307 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:24.582555 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:24.583335 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:24.583341 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:24.583388 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:24.583870 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:24.583883 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.585172 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.585179 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:24.586799 (+ 1620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:24.586814 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.588121 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.588128 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:24.588312 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.588336 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 13:03:24.588451 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 13:03:24.588597 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":770,"mutex_wait_us":1,"spinlock_wait_cycles":896,"thread_start_us":939,"threads_started":1}
I20260808 13:03:24.589181 4893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.581447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:34681 (local address 127.1.124.130:38055)
0808 13:03:24.582672 (+ 1225us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:24.582695 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:24.582850 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:24.583178 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:24.583183 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:24.583208 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:24.583718 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:24.583740 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.585167 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.585172 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:24.586847 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:24.586861 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.588289 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.588295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:24.588454 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.588477 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 13:03:24.588585 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 13:03:24.588715 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1082,"mutex_wait_us":48,"spinlock_wait_cycles":17792,"thread_start_us":503,"threads_started":1}
I20260808 13:03:24.590240 4370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be7ec46508e24730b1b6abaf83a81a67" candidate_uuid: "1019273dbea64a9ca91689ce685bc71a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" is_pre_election: true
I20260808 13:03:24.590533 4781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be7ec46508e24730b1b6abaf83a81a67" candidate_uuid: "1019273dbea64a9ca91689ce685bc71a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c182a7f66f6d49119a563266572c43b8" is_pre_election: true
I20260808 13:03:24.590891 4370 raft_consensus.cc:2454] T be7ec46508e24730b1b6abaf83a81a67 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1019273dbea64a9ca91689ce685bc71a in term 0.
I20260808 13:03:24.591079 4781 raft_consensus.cc:2454] T be7ec46508e24730b1b6abaf83a81a67 P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1019273dbea64a9ca91689ce685bc71a in term 0.
I20260808 13:03:24.591701 4472 leader_election.cc:302] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1019273dbea64a9ca91689ce685bc71a, c182a7f66f6d49119a563266572c43b8, fdde88d3d6dd4bf4a4bf7ca9ffbc4505; no voters:
I20260808 13:03:24.592186 4892 raft_consensus.cc:2781] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:03:24.592330 4892 raft_consensus.cc:482] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:03:24.592448 4892 raft_consensus.cc:3037] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:24.594821 4892 raft_consensus.cc:504] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:24.595475 4892 leader_election.cc:288] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a [CANDIDATE]: Term 1 election: Requested vote from peers fdde88d3d6dd4bf4a4bf7ca9ffbc4505 (127.1.124.129:37625), c182a7f66f6d49119a563266572c43b8 (127.1.124.131:34681)
I20260808 13:03:24.595999 4370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be7ec46508e24730b1b6abaf83a81a67" candidate_uuid: "1019273dbea64a9ca91689ce685bc71a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505"
I20260808 13:03:24.595986 4781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be7ec46508e24730b1b6abaf83a81a67" candidate_uuid: "1019273dbea64a9ca91689ce685bc71a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c182a7f66f6d49119a563266572c43b8"
I20260808 13:03:24.596261 4370 raft_consensus.cc:3037] T be7ec46508e24730b1b6abaf83a81a67 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:24.596287 4781 raft_consensus.cc:3037] T be7ec46508e24730b1b6abaf83a81a67 P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:24.598596 4370 raft_consensus.cc:2454] T be7ec46508e24730b1b6abaf83a81a67 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1019273dbea64a9ca91689ce685bc71a in term 1.
I20260808 13:03:24.598611 4781 raft_consensus.cc:2454] T be7ec46508e24730b1b6abaf83a81a67 P c182a7f66f6d49119a563266572c43b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1019273dbea64a9ca91689ce685bc71a in term 1.
I20260808 13:03:24.599050 4472 leader_election.cc:302] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1019273dbea64a9ca91689ce685bc71a, fdde88d3d6dd4bf4a4bf7ca9ffbc4505; no voters:
I20260808 13:03:24.599515 4892 raft_consensus.cc:2781] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:24.600040 4892 raft_consensus.cc:686] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a [term 1 LEADER]: Becoming Leader. State: Replica: 1019273dbea64a9ca91689ce685bc71a, State: Running, Role: LEADER
I20260808 13:03:24.600638 4892 consensus_queue.cc:237] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:24.605644 4124 catalog_manager.cc:5495] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a reported cstate change: term changed from 0 to 1, leader changed from <none> to 1019273dbea64a9ca91689ce685bc71a (127.1.124.130). New cstate: current_term: 1 leader_uuid: "1019273dbea64a9ca91689ce685bc71a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } health_report { overall_health: UNKNOWN } } }
W20260808 13:03:24.610073 4649 tablet.cc:2369] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:03:24.654639 4855 tablet.cc:2369] T be7ec46508e24730b1b6abaf83a81a67 P c182a7f66f6d49119a563266572c43b8: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 13:03:24 dist-test-slave-t3q3 krb5kdc[4059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194202, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:24.674770 4902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.658928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:37271 (local address 127.0.0.1:51600)
0808 13:03:24.659592 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:24.659611 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:24.659766 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:24.660298 (+ 532us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:24.660304 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:24.661224 (+ 920us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:24.661584 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:24.661596 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.662718 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.662723 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:24.664370 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:24.664383 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.665235 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.665242 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:24.665436 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.666162 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:24.666184 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:24.668882 (+ 2698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:24.671866 (+ 2984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:24.671878 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:24.671886 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:24.672399 (+ 513us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:24.672888 (+ 489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:24.672895 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:24.672898 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:24.673053 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:24.673623 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:24.673633 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:24.673826 (+ 193us) client_negotiation.cc:769] Sending connection context
0808 13:03:24.673992 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 13:03:24.674308 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":516,"thread_start_us":363,"threads_started":1}
I20260808 13:03:24.674795 4483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.658841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51600 (local address 127.1.124.190:37271)
0808 13:03:24.659098 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:24.659108 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:24.659708 (+ 600us) server_negotiation.cc:407] Connection header received
0808 13:03:24.659904 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:24.659909 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:24.659993 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:24.660260 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:24.661746 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.662588 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.664530 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.665037 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.665138 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.669073 (+ 3935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:24.669078 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:24.669081 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:24.669109 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:24.671724 (+ 2615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:24.672517 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:24.672524 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:24.672527 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:24.672773 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:24.673175 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:24.673182 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:24.673190 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:24.673480 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:24.673675 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:24.674123 (+ 448us) server_negotiation.cc:299] Negotiation successful
0808 13:03:24.674437 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":111,"server-negotiator.queue_time_us":66}
I20260808 13:03:24.680612 4124 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:51600:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 13:03:24.681579 4124 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:03:24.688436 4538 tablet_service.cc:1467] Processing CreateTablet for tablet 729949070cac4a29b5f5a08680ff9fb8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8b9a6c017a2e45e0b3b6dd0bc10754fb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 13:03:24.688424 4335 tablet_service.cc:1467] Processing CreateTablet for tablet 729949070cac4a29b5f5a08680ff9fb8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8b9a6c017a2e45e0b3b6dd0bc10754fb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 13:03:24.688776 4747 tablet_service.cc:1467] Processing CreateTablet for tablet 729949070cac4a29b5f5a08680ff9fb8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8b9a6c017a2e45e0b3b6dd0bc10754fb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 13:03:24.689453 4538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 729949070cac4a29b5f5a08680ff9fb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:24.689472 4335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 729949070cac4a29b5f5a08680ff9fb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:24.689592 4747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 729949070cac4a29b5f5a08680ff9fb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:24.695761 4882 tablet_bootstrap.cc:492] T 729949070cac4a29b5f5a08680ff9fb8 P 1019273dbea64a9ca91689ce685bc71a: Bootstrap starting.
I20260808 13:03:24.697018 4884 tablet_bootstrap.cc:492] T 729949070cac4a29b5f5a08680ff9fb8 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Bootstrap starting.
I20260808 13:03:24.697404 4882 tablet_bootstrap.cc:654] T 729949070cac4a29b5f5a08680ff9fb8 P 1019273dbea64a9ca91689ce685bc71a: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:24.698736 4884 tablet_bootstrap.cc:654] T 729949070cac4a29b5f5a08680ff9fb8 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:24.698768 4881 tablet_bootstrap.cc:492] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8: Bootstrap starting.
I20260808 13:03:24.699800 4882 tablet_bootstrap.cc:492] T 729949070cac4a29b5f5a08680ff9fb8 P 1019273dbea64a9ca91689ce685bc71a: No bootstrap required, opened a new log
I20260808 13:03:24.699944 4882 ts_tablet_manager.cc:1397] T 729949070cac4a29b5f5a08680ff9fb8 P 1019273dbea64a9ca91689ce685bc71a: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 13:03:24.700495 4881 tablet_bootstrap.cc:654] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:24.700857 4882 raft_consensus.cc:348] T 729949070cac4a29b5f5a08680ff9fb8 P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:24.701082 4882 raft_consensus.cc:374] T 729949070cac4a29b5f5a08680ff9fb8 P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:24.701174 4884 tablet_bootstrap.cc:492] T 729949070cac4a29b5f5a08680ff9fb8 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: No bootstrap required, opened a new log
I20260808 13:03:24.701184 4882 raft_consensus.cc:729] T 729949070cac4a29b5f5a08680ff9fb8 P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1019273dbea64a9ca91689ce685bc71a, State: Initialized, Role: FOLLOWER
I20260808 13:03:24.701326 4884 ts_tablet_manager.cc:1397] T 729949070cac4a29b5f5a08680ff9fb8 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 13:03:24.701442 4882 consensus_queue.cc:260] T 729949070cac4a29b5f5a08680ff9fb8 P 1019273dbea64a9ca91689ce685bc71a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:24.701874 4892 tablet_replica.cc:439] T 729949070cac4a29b5f5a08680ff9fb8 P 1019273dbea64a9ca91689ce685bc71a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } } }
I20260808 13:03:24.702047 4882 ts_tablet_manager.cc:1428] T 729949070cac4a29b5f5a08680ff9fb8 P 1019273dbea64a9ca91689ce685bc71a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 13:03:24.702194 4892 tablet_replica.cc:442] T 729949070cac4a29b5f5a08680ff9fb8 P 1019273dbea64a9ca91689ce685bc71a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:03:24.702306 4884 raft_consensus.cc:348] T 729949070cac4a29b5f5a08680ff9fb8 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:24.702558 4884 raft_consensus.cc:374] T 729949070cac4a29b5f5a08680ff9fb8 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:24.702673 4884 raft_consensus.cc:729] T 729949070cac4a29b5f5a08680ff9fb8 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdde88d3d6dd4bf4a4bf7ca9ffbc4505, State: Initialized, Role: FOLLOWER
I20260808 13:03:24.702932 4884 consensus_queue.cc:260] T 729949070cac4a29b5f5a08680ff9fb8 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:24.703562 4881 tablet_bootstrap.cc:492] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8: No bootstrap required, opened a new log
I20260808 13:03:24.703574 4884 ts_tablet_manager.cc:1428] T 729949070cac4a29b5f5a08680ff9fb8 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260808 13:03:24.703711 4881 ts_tablet_manager.cc:1397] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 13:03:24.703451 4891 tablet_replica.cc:439] T 729949070cac4a29b5f5a08680ff9fb8 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } } }
I20260808 13:03:24.703899 4891 tablet_replica.cc:442] T 729949070cac4a29b5f5a08680ff9fb8 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:03:24.704523 4881 raft_consensus.cc:348] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:24.704797 4881 raft_consensus.cc:374] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:24.704913 4881 raft_consensus.cc:729] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c182a7f66f6d49119a563266572c43b8, State: Initialized, Role: FOLLOWER
I20260808 13:03:24.705159 4881 consensus_queue.cc:260] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:24.705809 4881 ts_tablet_manager.cc:1428] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 13:03:24.705646 4890 tablet_replica.cc:439] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } } }
I20260808 13:03:24.706044 4890 tablet_replica.cc:442] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8: This TxnStatusTablet replica's current role is: FOLLOWER
W20260808 13:03:24.794126 4442 tablet.cc:2369] T 729949070cac4a29b5f5a08680ff9fb8 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:03:24.888878 4890 raft_consensus.cc:482] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:03:24.889092 4890 raft_consensus.cc:504] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:24.890760 4890 leader_election.cc:288] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fdde88d3d6dd4bf4a4bf7ca9ffbc4505 (127.1.124.129:37625), 1019273dbea64a9ca91689ce685bc71a (127.1.124.130:41619)
I20260808 13:03:24.899706 4878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.891230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:41939 (local address 127.1.124.130:41619)
0808 13:03:24.891476 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:24.891485 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:24.892047 (+ 562us) server_negotiation.cc:407] Connection header received
0808 13:03:24.892337 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:24.892345 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:24.892435 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:24.892554 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:24.893312 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.894405 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.897325 (+ 2920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.898581 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.898808 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.898892 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:24.899305 (+ 413us) server_negotiation.cc:299] Negotiation successful
0808 13:03:24.899436 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 13:03:24.899734 4877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.891076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:33819 (local address 127.1.124.129:37625)
0808 13:03:24.891313 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:24.891323 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:24.891903 (+ 580us) server_negotiation.cc:407] Connection header received
0808 13:03:24.892174 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:24.892180 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:24.892268 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:24.892402 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:24.893176 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.894380 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.897325 (+ 2945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.898581 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.898760 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.898838 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:24.899305 (+ 467us) server_negotiation.cc:299] Negotiation successful
0808 13:03:24.899437 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 13:03:24.899950 4908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.890979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:37625 (local address 127.1.124.131:33819)
0808 13:03:24.891771 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:24.891794 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:24.891954 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:24.892418 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:24.892422 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:24.892447 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:24.892905 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:24.892927 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.894560 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.894564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:24.897140 (+ 2576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:24.897156 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.898823 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.898828 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:24.899071 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.899160 (+ 89us) client_negotiation.cc:769] Sending connection context
0808 13:03:24.899262 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 13:03:24.899464 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"mutex_wait_us":251,"thread_start_us":515,"threads_started":1}
I20260808 13:03:24.899971 4911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.891087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:41619 (local address 127.1.124.131:41939)
0808 13:03:24.891934 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:24.891949 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:24.892173 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:24.892588 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:24.892592 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:24.892639 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:24.893054 (+ 415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:24.893065 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.894541 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.894548 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:24.897140 (+ 2592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:24.897156 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.898811 (+ 1655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.898817 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:24.899089 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.899164 (+ 75us) client_negotiation.cc:769] Sending connection context
0808 13:03:24.899262 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 13:03:24.899464 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":183,"thread_start_us":572,"threads_started":1}
I20260808 13:03:24.900686 4370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "729949070cac4a29b5f5a08680ff9fb8" candidate_uuid: "c182a7f66f6d49119a563266572c43b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" is_pre_election: true
I20260808 13:03:24.900995 4370 raft_consensus.cc:2454] T 729949070cac4a29b5f5a08680ff9fb8 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c182a7f66f6d49119a563266572c43b8 in term 0.
I20260808 13:03:24.900908 4576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "729949070cac4a29b5f5a08680ff9fb8" candidate_uuid: "c182a7f66f6d49119a563266572c43b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1019273dbea64a9ca91689ce685bc71a" is_pre_election: true
I20260808 13:03:24.901373 4576 raft_consensus.cc:2454] T 729949070cac4a29b5f5a08680ff9fb8 P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c182a7f66f6d49119a563266572c43b8 in term 0.
I20260808 13:03:24.901463 4677 leader_election.cc:302] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c182a7f66f6d49119a563266572c43b8, fdde88d3d6dd4bf4a4bf7ca9ffbc4505; no voters:
I20260808 13:03:24.901944 4890 raft_consensus.cc:2781] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:03:24.902088 4890 raft_consensus.cc:482] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:03:24.902191 4890 raft_consensus.cc:3037] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:24.904470 4890 raft_consensus.cc:504] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:24.905037 4890 leader_election.cc:288] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8 [CANDIDATE]: Term 1 election: Requested vote from peers fdde88d3d6dd4bf4a4bf7ca9ffbc4505 (127.1.124.129:37625), 1019273dbea64a9ca91689ce685bc71a (127.1.124.130:41619)
I20260808 13:03:24.905591 4576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "729949070cac4a29b5f5a08680ff9fb8" candidate_uuid: "c182a7f66f6d49119a563266572c43b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1019273dbea64a9ca91689ce685bc71a"
I20260808 13:03:24.905634 4370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "729949070cac4a29b5f5a08680ff9fb8" candidate_uuid: "c182a7f66f6d49119a563266572c43b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505"
I20260808 13:03:24.905750 4576 raft_consensus.cc:3037] T 729949070cac4a29b5f5a08680ff9fb8 P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:24.905887 4370 raft_consensus.cc:3037] T 729949070cac4a29b5f5a08680ff9fb8 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:24.908171 4370 raft_consensus.cc:2454] T 729949070cac4a29b5f5a08680ff9fb8 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c182a7f66f6d49119a563266572c43b8 in term 1.
I20260808 13:03:24.908183 4576 raft_consensus.cc:2454] T 729949070cac4a29b5f5a08680ff9fb8 P 1019273dbea64a9ca91689ce685bc71a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c182a7f66f6d49119a563266572c43b8 in term 1.
I20260808 13:03:24.908627 4677 leader_election.cc:302] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c182a7f66f6d49119a563266572c43b8, fdde88d3d6dd4bf4a4bf7ca9ffbc4505; no voters:
I20260808 13:03:24.908912 4890 raft_consensus.cc:2781] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:24.909313 4890 raft_consensus.cc:686] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8 [term 1 LEADER]: Becoming Leader. State: Replica: c182a7f66f6d49119a563266572c43b8, State: Running, Role: LEADER
I20260808 13:03:24.909825 4890 consensus_queue.cc:237] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:24.913312 4912 tablet_replica.cc:439] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8: TxnStatusTablet state changed. Reason: New leader c182a7f66f6d49119a563266572c43b8. Latest consensus state: current_term: 1 leader_uuid: "c182a7f66f6d49119a563266572c43b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } } }
I20260808 13:03:24.913585 4912 tablet_replica.cc:442] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8: This TxnStatusTablet replica's current role is: LEADER
I20260808 13:03:24.914338 4914 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 13:03:24.914656 4914 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 13:03:24.915262 4124 catalog_manager.cc:5495] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to c182a7f66f6d49119a563266572c43b8 (127.1.124.131). New cstate: current_term: 1 leader_uuid: "c182a7f66f6d49119a563266572c43b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } health_report { overall_health: HEALTHY } } }
I20260808 13:03:24.941486 4880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.935518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37062 (local address 127.1.124.131:34681)
0808 13:03:24.935724 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:24.935731 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:24.935916 (+ 185us) server_negotiation.cc:407] Connection header received
0808 13:03:24.936249 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:24.936253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:24.936331 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:24.936483 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:03:24.936887 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.937768 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.939133 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.939590 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.939740 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.940252 (+ 512us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:03:24.940784 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:03:24.940964 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:24.941033 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 13:03:24.941143 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 13:03:24.941461 4902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.935551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:34681 (local address 127.0.0.1:37062)
0808 13:03:24.935824 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:24.935841 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:24.936006 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:24.936533 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:24.936540 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:24.936555 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:03:24.936746 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:24.936754 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.937942 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.937947 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:24.938985 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:24.938997 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.939752 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.939757 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:24.939942 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.939999 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:03:24.940906 (+ 907us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:03:24.940928 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 13:03:24.941006 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 13:03:24.941161 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260808 13:03:24.949743 4576 raft_consensus.cc:1261] T 729949070cac4a29b5f5a08680ff9fb8 P 1019273dbea64a9ca91689ce685bc71a [term 1 FOLLOWER]: Refusing update from remote peer c182a7f66f6d49119a563266572c43b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:03:24.949735 4370 raft_consensus.cc:1261] T 729949070cac4a29b5f5a08680ff9fb8 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 1 FOLLOWER]: Refusing update from remote peer c182a7f66f6d49119a563266572c43b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:03:24.950785 4890 consensus_queue.cc:1035] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:03:24.951538 4890 consensus_queue.cc:1035] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:03:24.959126 4891 tablet_replica.cc:439] T 729949070cac4a29b5f5a08680ff9fb8 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: TxnStatusTablet state changed. Reason: New leader c182a7f66f6d49119a563266572c43b8. Latest consensus state: current_term: 1 leader_uuid: "c182a7f66f6d49119a563266572c43b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } } }
I20260808 13:03:24.959430 4891 tablet_replica.cc:442] T 729949070cac4a29b5f5a08680ff9fb8 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:03:24.963621 4920 mvcc.cc:205] Tried to move back new op lower bound from 7316251463463497728 to 7316251463314898944. Current Snapshot: MvccSnapshot[applied={T|T < 7316251463463497728}]
I20260808 13:03:24.964416 4891 tablet_replica.cc:439] T 729949070cac4a29b5f5a08680ff9fb8 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c182a7f66f6d49119a563266572c43b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } } }
I20260808 13:03:24.964660 4891 tablet_replica.cc:442] T 729949070cac4a29b5f5a08680ff9fb8 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:03:24.965683 4890 tablet_replica.cc:439] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c182a7f66f6d49119a563266572c43b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } } }
I20260808 13:03:24.966048 4890 tablet_replica.cc:442] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8: This TxnStatusTablet replica's current role is: LEADER
I20260808 13:03:24.966970 4918 mvcc.cc:205] Tried to move back new op lower bound from 7316251463463497728 to 7316251463314898944. Current Snapshot: MvccSnapshot[applied={T|T < 7316251463463497728}]
I20260808 13:03:24.976433 4892 tablet_replica.cc:439] T 729949070cac4a29b5f5a08680ff9fb8 P 1019273dbea64a9ca91689ce685bc71a: TxnStatusTablet state changed. Reason: New leader c182a7f66f6d49119a563266572c43b8. Latest consensus state: current_term: 1 leader_uuid: "c182a7f66f6d49119a563266572c43b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } } }
I20260808 13:03:24.976826 4892 tablet_replica.cc:442] T 729949070cac4a29b5f5a08680ff9fb8 P 1019273dbea64a9ca91689ce685bc71a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:03:24.979642 4912 tablet_replica.cc:439] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c182a7f66f6d49119a563266572c43b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } } }
I20260808 13:03:24.979923 4912 tablet_replica.cc:442] T 729949070cac4a29b5f5a08680ff9fb8 P c182a7f66f6d49119a563266572c43b8: This TxnStatusTablet replica's current role is: LEADER
I20260808 13:03:24.982662 4921 mvcc.cc:205] Tried to move back new op lower bound from 7316251463463497728 to 7316251463314898944. Current Snapshot: MvccSnapshot[applied={T|T < 7316251463463497728}]
I20260808 13:03:24.983848 4892 tablet_replica.cc:439] T 729949070cac4a29b5f5a08680ff9fb8 P 1019273dbea64a9ca91689ce685bc71a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c182a7f66f6d49119a563266572c43b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } } }
I20260808 13:03:24.984143 4892 tablet_replica.cc:442] T 729949070cac4a29b5f5a08680ff9fb8 P 1019273dbea64a9ca91689ce685bc71a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:03:24.989399 4878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:24.977307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37954 (local address 127.1.124.130:41619)
0808 13:03:24.977527 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:24.977536 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:24.978154 (+ 618us) server_negotiation.cc:407] Connection header received
0808 13:03:24.978481 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:24.978488 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:24.978633 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:24.978783 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:03:24.979601 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.980694 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.986444 (+ 5750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:24.987065 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:24.987193 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:24.987875 (+ 682us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:03:24.988633 (+ 758us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:03:24.988776 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:24.989009 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 13:03:24.989088 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 13:03:25.006353 4880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:25.000524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:46143 (local address 127.1.124.131:34681)
0808 13:03:25.000766 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:25.000776 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:25.001186 (+ 410us) server_negotiation.cc:407] Connection header received
0808 13:03:25.001420 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:25.001424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:25.001492 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:25.001586 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:03:25.002097 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:25.002922 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:25.004489 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:25.004959 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:25.005102 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:25.005553 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:03:25.005661 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:03:25.005764 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:25.005905 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 13:03:25.005993 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 13:03:25.006455 4935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:25.000370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:34681 (local address 127.1.124.130:46143)
0808 13:03:25.001070 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:25.001086 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:25.001265 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:25.001603 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:25.001608 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:25.001620 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:03:25.001949 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:25.001958 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:25.003057 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:25.003064 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:25.004308 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:25.004338 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:25.005156 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:25.005165 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:25.005296 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:25.005349 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:03:25.005765 (+ 416us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:03:25.005779 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 13:03:25.005883 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 13:03:25.006014 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":387,"threads_started":1}
I20260808 13:03:25.016880 4370 raft_consensus.cc:1261] T be7ec46508e24730b1b6abaf83a81a67 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 1 FOLLOWER]: Refusing update from remote peer 1019273dbea64a9ca91689ce685bc71a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:03:25.017040 4781 raft_consensus.cc:1261] T be7ec46508e24730b1b6abaf83a81a67 P c182a7f66f6d49119a563266572c43b8 [term 1 FOLLOWER]: Refusing update from remote peer 1019273dbea64a9ca91689ce685bc71a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:03:25.017668 4895 consensus_queue.cc:1035] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:03:25.018353 4892 consensus_queue.cc:1035] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:03:25.057463 4878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:25.049889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:38023 (local address 127.1.124.130:41619)
0808 13:03:25.050065 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:25.050074 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:25.050772 (+ 698us) server_negotiation.cc:407] Connection header received
0808 13:03:25.050994 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:25.050999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:25.051077 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:25.051183 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:03:25.051953 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:25.053142 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:25.055343 (+ 2201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:25.055991 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:25.056177 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:25.056683 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:03:25.056817 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:03:25.056912 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:25.057092 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 13:03:25.057178 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 13:03:25.057652 4941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:25.049928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:41619 (local address 127.1.124.131:38023)
0808 13:03:25.050664 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:25.050680 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:25.050849 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:25.051360 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:25.051366 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:25.051379 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:03:25.051782 (+ 403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:25.051792 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:25.053308 (+ 1516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:25.053313 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:25.055038 (+ 1725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:25.055055 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:25.056243 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:25.056248 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:25.056440 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:25.056499 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:03:25.056919 (+ 420us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:03:25.056967 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 13:03:25.057049 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 13:03:25.057246 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":395,"threads_started":1}
I20260808 13:03:25.103317 4123 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:03:25.103652 4123 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:03:25.107852 4123 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7: Sending DeleteTablet for 3 replicas of tablet be7ec46508e24730b1b6abaf83a81a67
I20260808 13:03:25.109112 4335 tablet_service.cc:1514] Processing DeleteTablet for tablet be7ec46508e24730b1b6abaf83a81a67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:25 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:40020
I20260808 13:03:25.110037 4947 tablet_replica.cc:330] T be7ec46508e24730b1b6abaf83a81a67 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: stopping tablet replica
I20260808 13:03:25.110179 4538 tablet_service.cc:1514] Processing DeleteTablet for tablet be7ec46508e24730b1b6abaf83a81a67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:25 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:37938
I20260808 13:03:25.110487 4947 raft_consensus.cc:2229] T be7ec46508e24730b1b6abaf83a81a67 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:03:25.111017 4947 raft_consensus.cc:2258] T be7ec46508e24730b1b6abaf83a81a67 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:25.110981 4747 tablet_service.cc:1514] Processing DeleteTablet for tablet be7ec46508e24730b1b6abaf83a81a67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:25 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:37056
I20260808 13:03:25.112120 4948 tablet_replica.cc:330] T be7ec46508e24730b1b6abaf83a81a67 P c182a7f66f6d49119a563266572c43b8: stopping tablet replica
I20260808 13:03:25.112541 4948 raft_consensus.cc:2229] T be7ec46508e24730b1b6abaf83a81a67 P c182a7f66f6d49119a563266572c43b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:03:25.112874 4948 raft_consensus.cc:2258] T be7ec46508e24730b1b6abaf83a81a67 P c182a7f66f6d49119a563266572c43b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:25.113564 4123 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51584:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:03:25.114288 4123 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:03:25.119601 4948 ts_tablet_manager.cc:1905] T be7ec46508e24730b1b6abaf83a81a67 P c182a7f66f6d49119a563266572c43b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:25.120369 4947 ts_tablet_manager.cc:1905] T be7ec46508e24730b1b6abaf83a81a67 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:25.123368 4747 tablet_service.cc:1467] Processing CreateTablet for tablet 8a60f98e84404778b5844244d2c341ab (DEFAULT_TABLE table=test-table [id=53a2b503de16431d9295e1b976d515ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:25.123381 4537 tablet_service.cc:1467] Processing CreateTablet for tablet 8a60f98e84404778b5844244d2c341ab (DEFAULT_TABLE table=test-table [id=53a2b503de16431d9295e1b976d515ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:25.123829 4747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a60f98e84404778b5844244d2c341ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:25.123891 4537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a60f98e84404778b5844244d2c341ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:25.127768 4947 ts_tablet_manager.cc:1918] T be7ec46508e24730b1b6abaf83a81a67 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:03:25.127924 4947 log.cc:1196] T be7ec46508e24730b1b6abaf83a81a67 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/be7ec46508e24730b1b6abaf83a81a67
I20260808 13:03:25.127996 4948 ts_tablet_manager.cc:1918] T be7ec46508e24730b1b6abaf83a81a67 P c182a7f66f6d49119a563266572c43b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:03:25.128206 4948 log.cc:1196] T be7ec46508e24730b1b6abaf83a81a67 P c182a7f66f6d49119a563266572c43b8: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/be7ec46508e24730b1b6abaf83a81a67
I20260808 13:03:25.128569 4947 ts_tablet_manager.cc:1939] T be7ec46508e24730b1b6abaf83a81a67 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Deleting consensus metadata
I20260808 13:03:25.128716 4948 ts_tablet_manager.cc:1939] T be7ec46508e24730b1b6abaf83a81a67 P c182a7f66f6d49119a563266572c43b8: Deleting consensus metadata
I20260808 13:03:25.129559 4335 tablet_service.cc:1467] Processing CreateTablet for tablet 8a60f98e84404778b5844244d2c341ab (DEFAULT_TABLE table=test-table [id=53a2b503de16431d9295e1b976d515ad]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:25.130014 4335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a60f98e84404778b5844244d2c341ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:25.130473 4882 tablet_bootstrap.cc:492] T 8a60f98e84404778b5844244d2c341ab P 1019273dbea64a9ca91689ce685bc71a: Bootstrap starting.
I20260808 13:03:25.130415 4103 catalog_manager.cc:4841] TS c182a7f66f6d49119a563266572c43b8 (127.1.124.131:34681): tablet be7ec46508e24730b1b6abaf83a81a67 (table test-table [id=d27f86e2f0b54f65b0d687c2888ab668]) successfully deleted
I20260808 13:03:25.130790 4102 catalog_manager.cc:4841] TS fdde88d3d6dd4bf4a4bf7ca9ffbc4505 (127.1.124.129:37625): tablet be7ec46508e24730b1b6abaf83a81a67 (table test-table [id=d27f86e2f0b54f65b0d687c2888ab668]) successfully deleted
I20260808 13:03:25.132369 4882 tablet_bootstrap.cc:654] T 8a60f98e84404778b5844244d2c341ab P 1019273dbea64a9ca91689ce685bc71a: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:25.147326 4882 tablet_bootstrap.cc:492] T 8a60f98e84404778b5844244d2c341ab P 1019273dbea64a9ca91689ce685bc71a: No bootstrap required, opened a new log
I20260808 13:03:25.147563 4882 ts_tablet_manager.cc:1397] T 8a60f98e84404778b5844244d2c341ab P 1019273dbea64a9ca91689ce685bc71a: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.012s
I20260808 13:03:25.148592 4882 raft_consensus.cc:348] T 8a60f98e84404778b5844244d2c341ab P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:25.148895 4882 raft_consensus.cc:374] T 8a60f98e84404778b5844244d2c341ab P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:25.148986 4882 raft_consensus.cc:729] T 8a60f98e84404778b5844244d2c341ab P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1019273dbea64a9ca91689ce685bc71a, State: Initialized, Role: FOLLOWER
I20260808 13:03:25.149470 4884 tablet_bootstrap.cc:492] T 8a60f98e84404778b5844244d2c341ab P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Bootstrap starting.
I20260808 13:03:25.149334 4882 consensus_queue.cc:260] T 8a60f98e84404778b5844244d2c341ab P 1019273dbea64a9ca91689ce685bc71a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:25.149976 4882 ts_tablet_manager.cc:1428] T 8a60f98e84404778b5844244d2c341ab P 1019273dbea64a9ca91689ce685bc71a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 13:03:25.151371 4884 tablet_bootstrap.cc:654] T 8a60f98e84404778b5844244d2c341ab P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:25.155299 4881 tablet_bootstrap.cc:492] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8: Bootstrap starting.
I20260808 13:03:25.158104 4881 tablet_bootstrap.cc:654] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:25.160264 4949 tablet_replica.cc:330] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a: stopping tablet replica
I20260808 13:03:25.160797 4949 raft_consensus.cc:2229] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:03:25.161621 4949 raft_consensus.cc:2258] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:25.164217 4949 ts_tablet_manager.cc:1905] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:25.166160 4884 tablet_bootstrap.cc:492] T 8a60f98e84404778b5844244d2c341ab P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: No bootstrap required, opened a new log
I20260808 13:03:25.166714 4884 ts_tablet_manager.cc:1397] T 8a60f98e84404778b5844244d2c341ab P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Time spent bootstrapping tablet: real 0.017s user 0.004s sys 0.000s
I20260808 13:03:25.167577 4884 raft_consensus.cc:348] T 8a60f98e84404778b5844244d2c341ab P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:25.168411 4884 raft_consensus.cc:374] T 8a60f98e84404778b5844244d2c341ab P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:25.168887 4884 raft_consensus.cc:729] T 8a60f98e84404778b5844244d2c341ab P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdde88d3d6dd4bf4a4bf7ca9ffbc4505, State: Initialized, Role: FOLLOWER
I20260808 13:03:25.169302 4884 consensus_queue.cc:260] T 8a60f98e84404778b5844244d2c341ab P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:25.172343 4949 ts_tablet_manager.cc:1918] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:03:25.172530 4884 ts_tablet_manager.cc:1428] T 8a60f98e84404778b5844244d2c341ab P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260808 13:03:25.172533 4949 log.cc:1196] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/be7ec46508e24730b1b6abaf83a81a67
I20260808 13:03:25.173207 4949 ts_tablet_manager.cc:1939] T be7ec46508e24730b1b6abaf83a81a67 P 1019273dbea64a9ca91689ce685bc71a: Deleting consensus metadata
I20260808 13:03:25.175061 4103 catalog_manager.cc:4841] TS 1019273dbea64a9ca91689ce685bc71a (127.1.124.130:41619): tablet be7ec46508e24730b1b6abaf83a81a67 (table test-table [id=d27f86e2f0b54f65b0d687c2888ab668]) successfully deleted
I20260808 13:03:25.177414 4881 tablet_bootstrap.cc:492] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8: No bootstrap required, opened a new log
I20260808 13:03:25.177632 4881 ts_tablet_manager.cc:1397] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8: Time spent bootstrapping tablet: real 0.022s user 0.005s sys 0.003s
I20260808 13:03:25.178450 4881 raft_consensus.cc:348] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:25.178730 4881 raft_consensus.cc:374] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:25.178828 4881 raft_consensus.cc:729] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c182a7f66f6d49119a563266572c43b8, State: Initialized, Role: FOLLOWER
I20260808 13:03:25.179178 4881 consensus_queue.cc:260] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:25.180369 4881 ts_tablet_manager.cc:1428] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260808 13:03:25.384198 4890 raft_consensus.cc:482] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:03:25.384421 4890 raft_consensus.cc:504] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:25.385083 4890 leader_election.cc:288] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fdde88d3d6dd4bf4a4bf7ca9ffbc4505 (127.1.124.129:37625), 1019273dbea64a9ca91689ce685bc71a (127.1.124.130:41619)
I20260808 13:03:25.385736 4370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a60f98e84404778b5844244d2c341ab" candidate_uuid: "c182a7f66f6d49119a563266572c43b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" is_pre_election: true
I20260808 13:03:25.385911 4576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a60f98e84404778b5844244d2c341ab" candidate_uuid: "c182a7f66f6d49119a563266572c43b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1019273dbea64a9ca91689ce685bc71a" is_pre_election: true
I20260808 13:03:25.386073 4370 raft_consensus.cc:2454] T 8a60f98e84404778b5844244d2c341ab P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c182a7f66f6d49119a563266572c43b8 in term 0.
I20260808 13:03:25.386171 4576 raft_consensus.cc:2454] T 8a60f98e84404778b5844244d2c341ab P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c182a7f66f6d49119a563266572c43b8 in term 0.
I20260808 13:03:25.386629 4678 leader_election.cc:302] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1019273dbea64a9ca91689ce685bc71a, c182a7f66f6d49119a563266572c43b8; no voters:
I20260808 13:03:25.387086 4890 raft_consensus.cc:2781] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:03:25.387184 4890 raft_consensus.cc:482] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:03:25.387297 4890 raft_consensus.cc:3037] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:25.390471 4890 raft_consensus.cc:504] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:25.391708 4576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a60f98e84404778b5844244d2c341ab" candidate_uuid: "c182a7f66f6d49119a563266572c43b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1019273dbea64a9ca91689ce685bc71a"
I20260808 13:03:25.391940 4576 raft_consensus.cc:3037] T 8a60f98e84404778b5844244d2c341ab P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:25.392220 4370 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a60f98e84404778b5844244d2c341ab" candidate_uuid: "c182a7f66f6d49119a563266572c43b8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505"
I20260808 13:03:25.392452 4370 raft_consensus.cc:3037] T 8a60f98e84404778b5844244d2c341ab P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:25.395198 4576 raft_consensus.cc:2454] T 8a60f98e84404778b5844244d2c341ab P 1019273dbea64a9ca91689ce685bc71a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c182a7f66f6d49119a563266572c43b8 in term 1.
I20260808 13:03:25.395637 4370 raft_consensus.cc:2454] T 8a60f98e84404778b5844244d2c341ab P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c182a7f66f6d49119a563266572c43b8 in term 1.
I20260808 13:03:25.395668 4678 leader_election.cc:302] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1019273dbea64a9ca91689ce685bc71a, c182a7f66f6d49119a563266572c43b8; no voters:
I20260808 13:03:25.396014 4929 raft_consensus.cc:2781] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:25.396198 4929 raft_consensus.cc:686] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8 [term 1 LEADER]: Becoming Leader. State: Replica: c182a7f66f6d49119a563266572c43b8, State: Running, Role: LEADER
I20260808 13:03:25.396613 4929 consensus_queue.cc:237] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:25.397097 4890 leader_election.cc:288] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8 [CANDIDATE]: Term 1 election: Requested vote from peers fdde88d3d6dd4bf4a4bf7ca9ffbc4505 (127.1.124.129:37625), 1019273dbea64a9ca91689ce685bc71a (127.1.124.130:41619)
I20260808 13:03:25.401396 4123 catalog_manager.cc:5495] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8 reported cstate change: term changed from 0 to 1, leader changed from <none> to c182a7f66f6d49119a563266572c43b8 (127.1.124.131). New cstate: current_term: 1 leader_uuid: "c182a7f66f6d49119a563266572c43b8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } health_report { overall_health: HEALTHY } } }
I20260808 13:03:25.460497 4880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:25.454583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37074 (local address 127.1.124.131:34681)
0808 13:03:25.454981 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:25.454985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:25.454998 (+ 13us) server_negotiation.cc:407] Connection header received
0808 13:03:25.455045 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:25.455049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:25.455109 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:25.455176 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:03:25.455671 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:25.456518 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:25.458632 (+ 2114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:25.459071 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:25.459181 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:25.459711 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:03:25.459814 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:03:25.459877 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:25.460125 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 13:03:25.460201 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301}
I20260808 13:03:25.475111 4941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:25.470175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:34681 (local address 127.1.124.131:40045)
0808 13:03:25.470384 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:25.470403 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:25.470554 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:25.470849 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:25.470853 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:25.470863 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:03:25.471105 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:25.471116 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:25.472151 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:25.472156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:25.473265 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:25.473277 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:25.473955 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:25.473960 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:25.474126 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:25.474161 (+ 35us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:03:25.474510 (+ 349us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:03:25.474521 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 13:03:25.474599 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 13:03:25.474790 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":57}
I20260808 13:03:25.475118 4880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:25.470243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:40045 (local address 127.1.124.131:34681)
0808 13:03:25.470401 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:25.470407 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:25.470483 (+ 76us) server_negotiation.cc:407] Connection header received
0808 13:03:25.470680 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:25.470685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:25.470747 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:25.470838 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:03:25.471235 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:25.471949 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:25.473407 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:25.473812 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:25.473945 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:25.474317 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:03:25.474415 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:03:25.474511 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:25.474616 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 13:03:25.474731 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":61,"server-negotiator.queue_time_us":69}
I20260808 13:03:25.484930 4370 raft_consensus.cc:1261] T 8a60f98e84404778b5844244d2c341ab P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 1 FOLLOWER]: Refusing update from remote peer c182a7f66f6d49119a563266572c43b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:03:25.485029 4576 raft_consensus.cc:1261] T 8a60f98e84404778b5844244d2c341ab P 1019273dbea64a9ca91689ce685bc71a [term 1 FOLLOWER]: Refusing update from remote peer c182a7f66f6d49119a563266572c43b8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:03:25.485589 4929 consensus_queue.cc:1035] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:03:25.485925 4912 consensus_queue.cc:1035] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:03:25.532536 4918 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 13:03:27.137966 4123 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:03:27.138226 4123 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51584:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:03:27.142409 4123 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7: Sending DeleteTablet for 3 replicas of tablet 8a60f98e84404778b5844244d2c341ab
I20260808 13:03:27.143340 4335 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a60f98e84404778b5844244d2c341ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:27 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:40020
I20260808 13:03:27.143518 4538 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a60f98e84404778b5844244d2c341ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:27 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:37938
I20260808 13:03:27.143502 4743 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a60f98e84404778b5844244d2c341ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:27 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:37056
I20260808 13:03:27.155887 4988 tablet_replica.cc:330] T 8a60f98e84404778b5844244d2c341ab P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: stopping tablet replica
I20260808 13:03:27.156437 4988 raft_consensus.cc:2229] T 8a60f98e84404778b5844244d2c341ab P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:03:27.156885 4988 raft_consensus.cc:2258] T 8a60f98e84404778b5844244d2c341ab P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:27.158037 4990 tablet_replica.cc:330] T 8a60f98e84404778b5844244d2c341ab P 1019273dbea64a9ca91689ce685bc71a: stopping tablet replica
I20260808 13:03:27.158288 4990 raft_consensus.cc:2229] T 8a60f98e84404778b5844244d2c341ab P 1019273dbea64a9ca91689ce685bc71a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:03:27.158339 4988 ts_tablet_manager.cc:1905] T 8a60f98e84404778b5844244d2c341ab P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:27.158388 4989 tablet_replica.cc:330] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8: stopping tablet replica
I20260808 13:03:27.158612 4989 raft_consensus.cc:2229] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:03:27.158653 4990 raft_consensus.cc:2258] T 8a60f98e84404778b5844244d2c341ab P 1019273dbea64a9ca91689ce685bc71a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:27.159614 4989 raft_consensus.cc:2258] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:27.159863 4990 ts_tablet_manager.cc:1905] T 8a60f98e84404778b5844244d2c341ab P 1019273dbea64a9ca91689ce685bc71a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:27.160751 4989 ts_tablet_manager.cc:1905] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:27.166270 4988 ts_tablet_manager.cc:1918] T 8a60f98e84404778b5844244d2c341ab P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 13:03:27.166514 4988 log.cc:1196] T 8a60f98e84404778b5844244d2c341ab P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/8a60f98e84404778b5844244d2c341ab
I20260808 13:03:27.166589 4990 ts_tablet_manager.cc:1918] T 8a60f98e84404778b5844244d2c341ab P 1019273dbea64a9ca91689ce685bc71a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 13:03:27.166765 4990 log.cc:1196] T 8a60f98e84404778b5844244d2c341ab P 1019273dbea64a9ca91689ce685bc71a: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/8a60f98e84404778b5844244d2c341ab
I20260808 13:03:27.167142 4988 ts_tablet_manager.cc:1939] T 8a60f98e84404778b5844244d2c341ab P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Deleting consensus metadata
I20260808 13:03:27.167403 4989 ts_tablet_manager.cc:1918] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 13:03:27.167490 4990 ts_tablet_manager.cc:1939] T 8a60f98e84404778b5844244d2c341ab P 1019273dbea64a9ca91689ce685bc71a: Deleting consensus metadata
I20260808 13:03:27.167655 4989 log.cc:1196] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/8a60f98e84404778b5844244d2c341ab
I20260808 13:03:27.168255 4989 ts_tablet_manager.cc:1939] T 8a60f98e84404778b5844244d2c341ab P c182a7f66f6d49119a563266572c43b8: Deleting consensus metadata
I20260808 13:03:27.168572 4102 catalog_manager.cc:4841] TS fdde88d3d6dd4bf4a4bf7ca9ffbc4505 (127.1.124.129:37625): tablet 8a60f98e84404778b5844244d2c341ab (table test-table [id=53a2b503de16431d9295e1b976d515ad]) successfully deleted
I20260808 13:03:27.168721 4103 catalog_manager.cc:4841] TS 1019273dbea64a9ca91689ce685bc71a (127.1.124.130:41619): tablet 8a60f98e84404778b5844244d2c341ab (table test-table [id=53a2b503de16431d9295e1b976d515ad]) successfully deleted
I20260808 13:03:27.169445 4103 catalog_manager.cc:4841] TS c182a7f66f6d49119a563266572c43b8 (127.1.124.131:34681): tablet 8a60f98e84404778b5844244d2c341ab (table test-table [id=53a2b503de16431d9295e1b976d515ad]) successfully deleted
Aug 08 13:03:27 dist-test-slave-t3q3 krb5kdc[4059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194207, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 13:03:27 dist-test-slave-t3q3 krb5kdc[4059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194207, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:27.207886 5002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:27.192231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51612 (local address 127.1.124.190:37271)
0808 13:03:27.192818 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:27.192823 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:27.192936 (+ 113us) server_negotiation.cc:407] Connection header received
0808 13:03:27.193163 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:27.193168 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:27.193254 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:27.193393 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:27.194849 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:27.195624 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:27.197135 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:27.197542 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:27.197649 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:27.202101 (+ 4452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:27.202106 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:27.202110 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:27.202153 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:27.205240 (+ 3087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:27.205868 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:27.205873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:27.205875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:27.206006 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:27.206389 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:27.206393 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:27.206396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:27.206748 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:27.206936 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:27.207302 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 13:03:27.207524 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":285,"threads_started":1}
I20260808 13:03:27.211853 4123 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51612:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:03:27.212596 4123 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:03:27.219620 4335 tablet_service.cc:1467] Processing CreateTablet for tablet ec9718acad6948dfb3f0ba242a2f6a6e (DEFAULT_TABLE table=test-table [id=e144fcc93ffb4913b4c21f191a9907f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:27.219792 4538 tablet_service.cc:1467] Processing CreateTablet for tablet ec9718acad6948dfb3f0ba242a2f6a6e (DEFAULT_TABLE table=test-table [id=e144fcc93ffb4913b4c21f191a9907f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:27.220145 4335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec9718acad6948dfb3f0ba242a2f6a6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:27.220293 4538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec9718acad6948dfb3f0ba242a2f6a6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:27.220274 4743 tablet_service.cc:1467] Processing CreateTablet for tablet ec9718acad6948dfb3f0ba242a2f6a6e (DEFAULT_TABLE table=test-table [id=e144fcc93ffb4913b4c21f191a9907f6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:27.220773 4743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ec9718acad6948dfb3f0ba242a2f6a6e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:27.225996 5004 tablet_bootstrap.cc:492] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Bootstrap starting.
I20260808 13:03:27.227427 5005 tablet_bootstrap.cc:492] T ec9718acad6948dfb3f0ba242a2f6a6e P 1019273dbea64a9ca91689ce685bc71a: Bootstrap starting.
I20260808 13:03:27.228106 5004 tablet_bootstrap.cc:654] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:27.229408 5005 tablet_bootstrap.cc:654] T ec9718acad6948dfb3f0ba242a2f6a6e P 1019273dbea64a9ca91689ce685bc71a: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:27.232403 5008 tablet_bootstrap.cc:492] T ec9718acad6948dfb3f0ba242a2f6a6e P c182a7f66f6d49119a563266572c43b8: Bootstrap starting.
I20260808 13:03:27.232426 5005 tablet_bootstrap.cc:492] T ec9718acad6948dfb3f0ba242a2f6a6e P 1019273dbea64a9ca91689ce685bc71a: No bootstrap required, opened a new log
I20260808 13:03:27.232592 5005 ts_tablet_manager.cc:1397] T ec9718acad6948dfb3f0ba242a2f6a6e P 1019273dbea64a9ca91689ce685bc71a: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 13:03:27.233644 5005 raft_consensus.cc:348] T ec9718acad6948dfb3f0ba242a2f6a6e P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:27.233982 5005 raft_consensus.cc:374] T ec9718acad6948dfb3f0ba242a2f6a6e P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:27.234118 5005 raft_consensus.cc:729] T ec9718acad6948dfb3f0ba242a2f6a6e P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1019273dbea64a9ca91689ce685bc71a, State: Initialized, Role: FOLLOWER
I20260808 13:03:27.234187 5008 tablet_bootstrap.cc:654] T ec9718acad6948dfb3f0ba242a2f6a6e P c182a7f66f6d49119a563266572c43b8: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:27.234454 5005 consensus_queue.cc:260] T ec9718acad6948dfb3f0ba242a2f6a6e P 1019273dbea64a9ca91689ce685bc71a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:27.236002 5005 ts_tablet_manager.cc:1428] T ec9718acad6948dfb3f0ba242a2f6a6e P 1019273dbea64a9ca91689ce685bc71a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 13:03:27.236994 5004 tablet_bootstrap.cc:492] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: No bootstrap required, opened a new log
I20260808 13:03:27.237164 5004 ts_tablet_manager.cc:1397] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260808 13:03:27.237975 5008 tablet_bootstrap.cc:492] T ec9718acad6948dfb3f0ba242a2f6a6e P c182a7f66f6d49119a563266572c43b8: No bootstrap required, opened a new log
I20260808 13:03:27.238142 5008 ts_tablet_manager.cc:1397] T ec9718acad6948dfb3f0ba242a2f6a6e P c182a7f66f6d49119a563266572c43b8: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.002s
I20260808 13:03:27.238045 5004 raft_consensus.cc:348] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:27.238359 5004 raft_consensus.cc:374] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:27.238539 5004 raft_consensus.cc:729] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdde88d3d6dd4bf4a4bf7ca9ffbc4505, State: Initialized, Role: FOLLOWER
I20260808 13:03:27.238833 5004 consensus_queue.cc:260] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:27.239027 5008 raft_consensus.cc:348] T ec9718acad6948dfb3f0ba242a2f6a6e P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:27.239323 5008 raft_consensus.cc:374] T ec9718acad6948dfb3f0ba242a2f6a6e P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:27.239454 5008 raft_consensus.cc:729] T ec9718acad6948dfb3f0ba242a2f6a6e P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c182a7f66f6d49119a563266572c43b8, State: Initialized, Role: FOLLOWER
I20260808 13:03:27.239846 5004 ts_tablet_manager.cc:1428] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260808 13:03:27.239707 5008 consensus_queue.cc:260] T ec9718acad6948dfb3f0ba242a2f6a6e P c182a7f66f6d49119a563266572c43b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:27.240475 5008 ts_tablet_manager.cc:1428] T ec9718acad6948dfb3f0ba242a2f6a6e P c182a7f66f6d49119a563266572c43b8: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260808 13:03:27.432909 5015 raft_consensus.cc:482] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:03:27.433256 5015 raft_consensus.cc:504] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:27.434986 5015 leader_election.cc:288] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1019273dbea64a9ca91689ce685bc71a (127.1.124.130:41619), c182a7f66f6d49119a563266572c43b8 (127.1.124.131:34681)
I20260808 13:03:27.443166 5016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:27.435328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:41619 (local address 127.1.124.129:54723)
0808 13:03:27.435906 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:27.435932 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:27.436133 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:27.437158 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:27.437162 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:27.437186 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:27.437616 (+ 430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:27.437630 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:27.439070 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:27.439076 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:27.440932 (+ 1856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:27.440949 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:27.442306 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:27.442313 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:27.442510 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:27.442567 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 13:03:27.442678 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 13:03:27.442776 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":316,"threads_started":1}
I20260808 13:03:27.443547 5017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:27.435434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:54723 (local address 127.1.124.130:41619)
0808 13:03:27.436731 (+ 1297us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:27.436737 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:27.436762 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:03:27.436851 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:27.436858 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:27.436949 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:27.437128 (+ 179us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:27.437813 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:27.438923 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:27.441160 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:27.442100 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:27.442258 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:27.442312 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:27.442739 (+ 427us) server_negotiation.cc:299] Negotiation successful
0808 13:03:27.442924 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1137,"thread_start_us":975,"threads_started":1}
I20260808 13:03:27.443598 5018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:27.436202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:34681 (local address 127.1.124.129:44313)
0808 13:03:27.437250 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:27.437272 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:27.437405 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:27.437796 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:27.437800 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:27.437823 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:27.438355 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:27.438370 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:27.439651 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:27.439655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:27.441334 (+ 1679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:27.441348 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:27.442618 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:27.442623 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:27.442994 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:27.443029 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 13:03:27.443133 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 13:03:27.443257 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":894,"thread_start_us":671,"threads_started":1}
I20260808 13:03:27.443684 5020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:27.436333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:44313 (local address 127.1.124.131:34681)
0808 13:03:27.437005 (+ 672us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:27.437012 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:27.437371 (+ 359us) server_negotiation.cc:407] Connection header received
0808 13:03:27.437541 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:27.437546 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:27.437627 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:27.437751 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:27.438521 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:27.439505 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:27.441497 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:27.442361 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:27.442547 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:27.442616 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:27.443187 (+ 571us) server_negotiation.cc:299] Negotiation successful
0808 13:03:27.443337 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":502,"thread_start_us":322,"threads_started":1}
I20260808 13:03:27.444192 4573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec9718acad6948dfb3f0ba242a2f6a6e" candidate_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1019273dbea64a9ca91689ce685bc71a" is_pre_election: true
I20260808 13:03:27.444375 4781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec9718acad6948dfb3f0ba242a2f6a6e" candidate_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c182a7f66f6d49119a563266572c43b8" is_pre_election: true
I20260808 13:03:27.444483 4573 raft_consensus.cc:2454] T ec9718acad6948dfb3f0ba242a2f6a6e P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdde88d3d6dd4bf4a4bf7ca9ffbc4505 in term 0.
I20260808 13:03:27.444655 4781 raft_consensus.cc:2454] T ec9718acad6948dfb3f0ba242a2f6a6e P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdde88d3d6dd4bf4a4bf7ca9ffbc4505 in term 0.
I20260808 13:03:27.445133 4261 leader_election.cc:302] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1019273dbea64a9ca91689ce685bc71a, fdde88d3d6dd4bf4a4bf7ca9ffbc4505; no voters:
I20260808 13:03:27.445595 5015 raft_consensus.cc:2781] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:03:27.445730 5015 raft_consensus.cc:482] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:03:27.445830 5015 raft_consensus.cc:3037] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:27.448025 5015 raft_consensus.cc:504] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:27.449118 5015 leader_election.cc:288] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [CANDIDATE]: Term 1 election: Requested vote from peers 1019273dbea64a9ca91689ce685bc71a (127.1.124.130:41619), c182a7f66f6d49119a563266572c43b8 (127.1.124.131:34681)
I20260808 13:03:27.449560 4573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec9718acad6948dfb3f0ba242a2f6a6e" candidate_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1019273dbea64a9ca91689ce685bc71a"
I20260808 13:03:27.449787 4573 raft_consensus.cc:3037] T ec9718acad6948dfb3f0ba242a2f6a6e P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:27.449950 4781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ec9718acad6948dfb3f0ba242a2f6a6e" candidate_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c182a7f66f6d49119a563266572c43b8"
I20260808 13:03:27.450172 4781 raft_consensus.cc:3037] T ec9718acad6948dfb3f0ba242a2f6a6e P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:27.452018 4573 raft_consensus.cc:2454] T ec9718acad6948dfb3f0ba242a2f6a6e P 1019273dbea64a9ca91689ce685bc71a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdde88d3d6dd4bf4a4bf7ca9ffbc4505 in term 1.
I20260808 13:03:27.452224 4781 raft_consensus.cc:2454] T ec9718acad6948dfb3f0ba242a2f6a6e P c182a7f66f6d49119a563266572c43b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdde88d3d6dd4bf4a4bf7ca9ffbc4505 in term 1.
I20260808 13:03:27.452531 4261 leader_election.cc:302] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1019273dbea64a9ca91689ce685bc71a, fdde88d3d6dd4bf4a4bf7ca9ffbc4505; no voters:
I20260808 13:03:27.452842 5015 raft_consensus.cc:2781] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:27.453362 5015 raft_consensus.cc:686] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 1 LEADER]: Becoming Leader. State: Replica: fdde88d3d6dd4bf4a4bf7ca9ffbc4505, State: Running, Role: LEADER
I20260808 13:03:27.453809 5015 consensus_queue.cc:237] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:27.457746 4124 catalog_manager.cc:5495] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 reported cstate change: term changed from 0 to 1, leader changed from <none> to fdde88d3d6dd4bf4a4bf7ca9ffbc4505 (127.1.124.129). New cstate: current_term: 1 leader_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } health_report { overall_health: UNKNOWN } } }
I20260808 13:03:27.535166 5028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:27.528824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40030 (local address 127.1.124.129:37625)
0808 13:03:27.529862 (+ 1038us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:27.529866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:27.529886 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:03:27.529967 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:27.529970 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:27.530045 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:27.530193 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:03:27.530579 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:27.531373 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:27.532775 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:27.533201 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:27.533293 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:27.533863 (+ 570us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:03:27.534478 (+ 615us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:03:27.534661 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:27.534737 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 13:03:27.534819 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":920,"thread_start_us":775,"threads_started":1}
I20260808 13:03:27.550459 5020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:27.544869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:38415 (local address 127.1.124.131:34681)
0808 13:03:27.545062 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:27.545067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:27.545592 (+ 525us) server_negotiation.cc:407] Connection header received
0808 13:03:27.545728 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:27.545731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:27.545799 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:27.545878 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:03:27.546297 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:27.546961 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:27.548489 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:27.548928 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:27.549023 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:27.549622 (+ 599us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:03:27.549756 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:03:27.549844 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:27.550024 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 13:03:27.550096 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 13:03:27.550462 5036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:27.544785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:34681 (local address 127.1.124.129:38415)
0808 13:03:27.545470 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:27.545486 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:27.545615 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:27.545902 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:27.545905 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:27.545914 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:03:27.546173 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:27.546180 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:27.547107 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:27.547112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:27.548352 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:27.548365 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:27.549109 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:27.549114 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:27.549325 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:27.549403 (+ 78us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:03:27.549897 (+ 494us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:03:27.549908 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 13:03:27.549989 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 13:03:27.550089 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":359,"threads_started":1}
I20260808 13:03:27.559471 4781 raft_consensus.cc:1261] T ec9718acad6948dfb3f0ba242a2f6a6e P c182a7f66f6d49119a563266572c43b8 [term 1 FOLLOWER]: Refusing update from remote peer fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:03:27.559510 4573 raft_consensus.cc:1261] T ec9718acad6948dfb3f0ba242a2f6a6e P 1019273dbea64a9ca91689ce685bc71a [term 1 FOLLOWER]: Refusing update from remote peer fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:03:27.560241 5021 consensus_queue.cc:1035] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:03:27.560650 5015 consensus_queue.cc:1035] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:03:27.588840 5028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:27.583307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:44723 (local address 127.1.124.129:37625)
0808 13:03:27.583461 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:27.583465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:27.583565 (+ 100us) server_negotiation.cc:407] Connection header received
0808 13:03:27.583709 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:27.583712 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:27.583769 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:27.583843 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:03:27.584460 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:27.585356 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:27.587110 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:27.587616 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:27.587713 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:27.588153 (+ 440us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:03:27.588248 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:03:27.588332 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:27.588497 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 13:03:27.588557 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 13:03:27.589062 5043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:27.582870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:37625 (local address 127.1.124.131:44723)
0808 13:03:27.583493 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:27.583509 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:27.583621 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:27.583922 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:27.583926 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:27.583937 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:03:27.584320 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:27.584330 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:27.585492 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:27.585499 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:27.586975 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:27.586989 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:27.587793 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:27.587798 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:27.587918 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:27.587955 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:03:27.588359 (+ 404us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:03:27.588371 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 13:03:27.588445 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 13:03:27.588598 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":334,"threads_started":1}
I20260808 13:03:27.622431 4123 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:03:27.622756 4123 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:03:27.626123 4123 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7: Sending DeleteTablet for 3 replicas of tablet ec9718acad6948dfb3f0ba242a2f6a6e
I20260808 13:03:27.627096 4743 tablet_service.cc:1514] Processing DeleteTablet for tablet ec9718acad6948dfb3f0ba242a2f6a6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:27 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:37056
I20260808 13:03:27.627115 4335 tablet_service.cc:1514] Processing DeleteTablet for tablet ec9718acad6948dfb3f0ba242a2f6a6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:27 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:40020
I20260808 13:03:27.627408 4988 tablet_replica.cc:330] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: stopping tablet replica
I20260808 13:03:27.627476 4538 tablet_service.cc:1514] Processing DeleteTablet for tablet ec9718acad6948dfb3f0ba242a2f6a6e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:27 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:37938
I20260808 13:03:27.627660 4988 raft_consensus.cc:2229] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:03:27.627785 4989 tablet_replica.cc:330] T ec9718acad6948dfb3f0ba242a2f6a6e P c182a7f66f6d49119a563266572c43b8: stopping tablet replica
I20260808 13:03:27.628217 4990 tablet_replica.cc:330] T ec9718acad6948dfb3f0ba242a2f6a6e P 1019273dbea64a9ca91689ce685bc71a: stopping tablet replica
I20260808 13:03:27.628252 4988 raft_consensus.cc:2258] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:27.628415 4990 raft_consensus.cc:2229] T ec9718acad6948dfb3f0ba242a2f6a6e P 1019273dbea64a9ca91689ce685bc71a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:03:27.628607 4990 raft_consensus.cc:2258] T ec9718acad6948dfb3f0ba242a2f6a6e P 1019273dbea64a9ca91689ce685bc71a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:27.627985 4989 raft_consensus.cc:2229] T ec9718acad6948dfb3f0ba242a2f6a6e P c182a7f66f6d49119a563266572c43b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:03:27.628841 4123 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51612:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 13:03:27.629148 4989 raft_consensus.cc:2258] T ec9718acad6948dfb3f0ba242a2f6a6e P c182a7f66f6d49119a563266572c43b8 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260808 13:03:27.629491 4123 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:03:27.629590 4988 ts_tablet_manager.cc:1905] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:27.630345 4990 ts_tablet_manager.cc:1905] T ec9718acad6948dfb3f0ba242a2f6a6e P 1019273dbea64a9ca91689ce685bc71a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:27.631976 4989 ts_tablet_manager.cc:1905] T ec9718acad6948dfb3f0ba242a2f6a6e P c182a7f66f6d49119a563266572c43b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:27.637405 4990 ts_tablet_manager.cc:1918] T ec9718acad6948dfb3f0ba242a2f6a6e P 1019273dbea64a9ca91689ce685bc71a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:03:27.637583 4990 log.cc:1196] T ec9718acad6948dfb3f0ba242a2f6a6e P 1019273dbea64a9ca91689ce685bc71a: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/ec9718acad6948dfb3f0ba242a2f6a6e
I20260808 13:03:27.637879 4988 ts_tablet_manager.cc:1918] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:03:27.638059 4988 log.cc:1196] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/ec9718acad6948dfb3f0ba242a2f6a6e
I20260808 13:03:27.638075 4990 ts_tablet_manager.cc:1939] T ec9718acad6948dfb3f0ba242a2f6a6e P 1019273dbea64a9ca91689ce685bc71a: Deleting consensus metadata
I20260808 13:03:27.638535 4988 ts_tablet_manager.cc:1939] T ec9718acad6948dfb3f0ba242a2f6a6e P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Deleting consensus metadata
I20260808 13:03:27.639029 4989 ts_tablet_manager.cc:1918] T ec9718acad6948dfb3f0ba242a2f6a6e P c182a7f66f6d49119a563266572c43b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:03:27.639201 4989 log.cc:1196] T ec9718acad6948dfb3f0ba242a2f6a6e P c182a7f66f6d49119a563266572c43b8: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/ec9718acad6948dfb3f0ba242a2f6a6e
I20260808 13:03:27.639242 4103 catalog_manager.cc:4841] TS 1019273dbea64a9ca91689ce685bc71a (127.1.124.130:41619): tablet ec9718acad6948dfb3f0ba242a2f6a6e (table test-table [id=e144fcc93ffb4913b4c21f191a9907f6]) successfully deleted
I20260808 13:03:27.639717 4989 ts_tablet_manager.cc:1939] T ec9718acad6948dfb3f0ba242a2f6a6e P c182a7f66f6d49119a563266572c43b8: Deleting consensus metadata
I20260808 13:03:27.639672 4102 catalog_manager.cc:4841] TS fdde88d3d6dd4bf4a4bf7ca9ffbc4505 (127.1.124.129:37625): tablet ec9718acad6948dfb3f0ba242a2f6a6e (table test-table [id=e144fcc93ffb4913b4c21f191a9907f6]) successfully deleted
I20260808 13:03:27.640844 4103 catalog_manager.cc:4841] TS c182a7f66f6d49119a563266572c43b8 (127.1.124.131:34681): tablet ec9718acad6948dfb3f0ba242a2f6a6e (table test-table [id=e144fcc93ffb4913b4c21f191a9907f6]) successfully deleted
I20260808 13:03:27.644783 4335 tablet_service.cc:1467] Processing CreateTablet for tablet 3d4e86b7e68f4df6a43689f1ad769e22 (DEFAULT_TABLE table=test-table [id=c0efcc4fc59e40c2b6909f3662ec1d69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:27.645236 4335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d4e86b7e68f4df6a43689f1ad769e22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:27.645248 4743 tablet_service.cc:1467] Processing CreateTablet for tablet 3d4e86b7e68f4df6a43689f1ad769e22 (DEFAULT_TABLE table=test-table [id=c0efcc4fc59e40c2b6909f3662ec1d69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:27.645226 4538 tablet_service.cc:1467] Processing CreateTablet for tablet 3d4e86b7e68f4df6a43689f1ad769e22 (DEFAULT_TABLE table=test-table [id=c0efcc4fc59e40c2b6909f3662ec1d69]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:27.645642 4743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d4e86b7e68f4df6a43689f1ad769e22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:27.645664 4538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3d4e86b7e68f4df6a43689f1ad769e22. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:27.652619 5008 tablet_bootstrap.cc:492] T 3d4e86b7e68f4df6a43689f1ad769e22 P c182a7f66f6d49119a563266572c43b8: Bootstrap starting.
I20260808 13:03:27.653208 5004 tablet_bootstrap.cc:492] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Bootstrap starting.
I20260808 13:03:27.654498 5008 tablet_bootstrap.cc:654] T 3d4e86b7e68f4df6a43689f1ad769e22 P c182a7f66f6d49119a563266572c43b8: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:27.655269 5004 tablet_bootstrap.cc:654] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:27.660910 5008 tablet_bootstrap.cc:492] T 3d4e86b7e68f4df6a43689f1ad769e22 P c182a7f66f6d49119a563266572c43b8: No bootstrap required, opened a new log
I20260808 13:03:27.661247 5008 ts_tablet_manager.cc:1397] T 3d4e86b7e68f4df6a43689f1ad769e22 P c182a7f66f6d49119a563266572c43b8: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.002s
I20260808 13:03:27.662294 5008 raft_consensus.cc:348] T 3d4e86b7e68f4df6a43689f1ad769e22 P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:27.662683 5008 raft_consensus.cc:374] T 3d4e86b7e68f4df6a43689f1ad769e22 P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:27.662846 5008 raft_consensus.cc:729] T 3d4e86b7e68f4df6a43689f1ad769e22 P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c182a7f66f6d49119a563266572c43b8, State: Initialized, Role: FOLLOWER
I20260808 13:03:27.663249 5008 consensus_queue.cc:260] T 3d4e86b7e68f4df6a43689f1ad769e22 P c182a7f66f6d49119a563266572c43b8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:27.664618 5008 ts_tablet_manager.cc:1428] T 3d4e86b7e68f4df6a43689f1ad769e22 P c182a7f66f6d49119a563266572c43b8: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s
I20260808 13:03:27.669821 5004 tablet_bootstrap.cc:492] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: No bootstrap required, opened a new log
I20260808 13:03:27.670357 5004 ts_tablet_manager.cc:1397] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.009s
I20260808 13:03:27.671300 5004 raft_consensus.cc:348] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:27.671584 5004 raft_consensus.cc:374] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:27.671690 5004 raft_consensus.cc:729] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdde88d3d6dd4bf4a4bf7ca9ffbc4505, State: Initialized, Role: FOLLOWER
I20260808 13:03:27.671943 5004 consensus_queue.cc:260] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:27.672360 5005 tablet_bootstrap.cc:492] T 3d4e86b7e68f4df6a43689f1ad769e22 P 1019273dbea64a9ca91689ce685bc71a: Bootstrap starting.
I20260808 13:03:27.672689 5004 ts_tablet_manager.cc:1428] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 13:03:27.674136 5005 tablet_bootstrap.cc:654] T 3d4e86b7e68f4df6a43689f1ad769e22 P 1019273dbea64a9ca91689ce685bc71a: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:27.677870 5005 tablet_bootstrap.cc:492] T 3d4e86b7e68f4df6a43689f1ad769e22 P 1019273dbea64a9ca91689ce685bc71a: No bootstrap required, opened a new log
I20260808 13:03:27.678087 5005 ts_tablet_manager.cc:1397] T 3d4e86b7e68f4df6a43689f1ad769e22 P 1019273dbea64a9ca91689ce685bc71a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 13:03:27.678846 5005 raft_consensus.cc:348] T 3d4e86b7e68f4df6a43689f1ad769e22 P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:27.679152 5005 raft_consensus.cc:374] T 3d4e86b7e68f4df6a43689f1ad769e22 P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:27.679275 5005 raft_consensus.cc:729] T 3d4e86b7e68f4df6a43689f1ad769e22 P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1019273dbea64a9ca91689ce685bc71a, State: Initialized, Role: FOLLOWER
I20260808 13:03:27.679575 5005 consensus_queue.cc:260] T 3d4e86b7e68f4df6a43689f1ad769e22 P 1019273dbea64a9ca91689ce685bc71a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:27.682158 5005 ts_tablet_manager.cc:1428] T 3d4e86b7e68f4df6a43689f1ad769e22 P 1019273dbea64a9ca91689ce685bc71a: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260808 13:03:27.800377 5021 raft_consensus.cc:482] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:03:27.800700 5021 raft_consensus.cc:504] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:27.801388 5021 leader_election.cc:288] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1019273dbea64a9ca91689ce685bc71a (127.1.124.130:41619), c182a7f66f6d49119a563266572c43b8 (127.1.124.131:34681)
I20260808 13:03:27.802084 4573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d4e86b7e68f4df6a43689f1ad769e22" candidate_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1019273dbea64a9ca91689ce685bc71a" is_pre_election: true
I20260808 13:03:27.802207 4781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d4e86b7e68f4df6a43689f1ad769e22" candidate_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c182a7f66f6d49119a563266572c43b8" is_pre_election: true
I20260808 13:03:27.802388 4573 raft_consensus.cc:2454] T 3d4e86b7e68f4df6a43689f1ad769e22 P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdde88d3d6dd4bf4a4bf7ca9ffbc4505 in term 0.
I20260808 13:03:27.802439 4781 raft_consensus.cc:2454] T 3d4e86b7e68f4df6a43689f1ad769e22 P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fdde88d3d6dd4bf4a4bf7ca9ffbc4505 in term 0.
I20260808 13:03:27.802994 4261 leader_election.cc:302] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c182a7f66f6d49119a563266572c43b8, fdde88d3d6dd4bf4a4bf7ca9ffbc4505; no voters:
I20260808 13:03:27.803301 5021 raft_consensus.cc:2781] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:03:27.803421 5021 raft_consensus.cc:482] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:03:27.803579 5021 raft_consensus.cc:3037] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:27.806141 5021 raft_consensus.cc:504] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:27.806893 5021 leader_election.cc:288] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [CANDIDATE]: Term 1 election: Requested vote from peers 1019273dbea64a9ca91689ce685bc71a (127.1.124.130:41619), c182a7f66f6d49119a563266572c43b8 (127.1.124.131:34681)
I20260808 13:03:27.807441 4781 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d4e86b7e68f4df6a43689f1ad769e22" candidate_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c182a7f66f6d49119a563266572c43b8"
I20260808 13:03:27.807469 4573 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3d4e86b7e68f4df6a43689f1ad769e22" candidate_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1019273dbea64a9ca91689ce685bc71a"
I20260808 13:03:27.807701 4781 raft_consensus.cc:3037] T 3d4e86b7e68f4df6a43689f1ad769e22 P c182a7f66f6d49119a563266572c43b8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:27.807727 4573 raft_consensus.cc:3037] T 3d4e86b7e68f4df6a43689f1ad769e22 P 1019273dbea64a9ca91689ce685bc71a [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:27.809965 4781 raft_consensus.cc:2454] T 3d4e86b7e68f4df6a43689f1ad769e22 P c182a7f66f6d49119a563266572c43b8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdde88d3d6dd4bf4a4bf7ca9ffbc4505 in term 1.
I20260808 13:03:27.810441 4261 leader_election.cc:302] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c182a7f66f6d49119a563266572c43b8, fdde88d3d6dd4bf4a4bf7ca9ffbc4505; no voters:
I20260808 13:03:27.810742 5021 raft_consensus.cc:2781] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:27.810961 5021 raft_consensus.cc:686] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 1 LEADER]: Becoming Leader. State: Replica: fdde88d3d6dd4bf4a4bf7ca9ffbc4505, State: Running, Role: LEADER
I20260808 13:03:27.811579 4573 raft_consensus.cc:2454] T 3d4e86b7e68f4df6a43689f1ad769e22 P 1019273dbea64a9ca91689ce685bc71a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fdde88d3d6dd4bf4a4bf7ca9ffbc4505 in term 1.
I20260808 13:03:27.811426 5021 consensus_queue.cc:237] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } }
I20260808 13:03:27.814697 4122 catalog_manager.cc:5495] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 reported cstate change: term changed from 0 to 1, leader changed from <none> to fdde88d3d6dd4bf4a4bf7ca9ffbc4505 (127.1.124.129). New cstate: current_term: 1 leader_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fdde88d3d6dd4bf4a4bf7ca9ffbc4505" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 37625 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 } health_report { overall_health: UNKNOWN } } }
I20260808 13:03:27.870285 4781 raft_consensus.cc:1261] T 3d4e86b7e68f4df6a43689f1ad769e22 P c182a7f66f6d49119a563266572c43b8 [term 1 FOLLOWER]: Refusing update from remote peer fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:03:27.870961 5021 consensus_queue.cc:1035] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c182a7f66f6d49119a563266572c43b8" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34681 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:03:27.871424 4573 raft_consensus.cc:1261] T 3d4e86b7e68f4df6a43689f1ad769e22 P 1019273dbea64a9ca91689ce685bc71a [term 1 FOLLOWER]: Refusing update from remote peer fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:03:27.871971 5021 consensus_queue.cc:1035] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1019273dbea64a9ca91689ce685bc71a" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41619 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:03:27.907652 4967 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 13:03:29.492246 4123 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:03:29.492479 4123 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51612:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:03:29.496713 4123 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67026e7bf983405485fc34735ed335a7: Sending DeleteTablet for 3 replicas of tablet 3d4e86b7e68f4df6a43689f1ad769e22
I20260808 13:03:29.497567 4538 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d4e86b7e68f4df6a43689f1ad769e22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:29 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:37938
I20260808 13:03:29.497682 4743 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d4e86b7e68f4df6a43689f1ad769e22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:29 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:37056
I20260808 13:03:29.497753 4335 tablet_service.cc:1514] Processing DeleteTablet for tablet 3d4e86b7e68f4df6a43689f1ad769e22 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:29 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:40020
I20260808 13:03:29.498350 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 4236
I20260808 13:03:29.510572 5086 tablet_replica.cc:330] T 3d4e86b7e68f4df6a43689f1ad769e22 P 1019273dbea64a9ca91689ce685bc71a: stopping tablet replica
I20260808 13:03:29.510843 5087 tablet_replica.cc:330] T 3d4e86b7e68f4df6a43689f1ad769e22 P c182a7f66f6d49119a563266572c43b8: stopping tablet replica
I20260808 13:03:29.510957 5086 raft_consensus.cc:2229] T 3d4e86b7e68f4df6a43689f1ad769e22 P 1019273dbea64a9ca91689ce685bc71a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:03:29.511106 5089 tablet_replica.cc:330] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: stopping tablet replica
I20260808 13:03:29.511234 5087 raft_consensus.cc:2229] T 3d4e86b7e68f4df6a43689f1ad769e22 P c182a7f66f6d49119a563266572c43b8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:03:29.511384 5086 raft_consensus.cc:2258] T 3d4e86b7e68f4df6a43689f1ad769e22 P 1019273dbea64a9ca91689ce685bc71a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:29.511418 5089 raft_consensus.cc:2229] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:03:29.511538 5087 raft_consensus.cc:2258] T 3d4e86b7e68f4df6a43689f1ad769e22 P c182a7f66f6d49119a563266572c43b8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:29.512028 5089 raft_consensus.cc:2258] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:29.513185 5086 ts_tablet_manager.cc:1905] T 3d4e86b7e68f4df6a43689f1ad769e22 P 1019273dbea64a9ca91689ce685bc71a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:29.513245 5089 ts_tablet_manager.cc:1905] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:29.513461 5087 ts_tablet_manager.cc:1905] T 3d4e86b7e68f4df6a43689f1ad769e22 P c182a7f66f6d49119a563266572c43b8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:29.520563 5086 ts_tablet_manager.cc:1918] T 3d4e86b7e68f4df6a43689f1ad769e22 P 1019273dbea64a9ca91689ce685bc71a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 13:03:29.520649 5089 ts_tablet_manager.cc:1918] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 13:03:29.520725 5087 ts_tablet_manager.cc:1918] T 3d4e86b7e68f4df6a43689f1ad769e22 P c182a7f66f6d49119a563266572c43b8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 13:03:29.520790 5086 log.cc:1196] T 3d4e86b7e68f4df6a43689f1ad769e22 P 1019273dbea64a9ca91689ce685bc71a: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/3d4e86b7e68f4df6a43689f1ad769e22
I20260808 13:03:29.520937 5087 log.cc:1196] T 3d4e86b7e68f4df6a43689f1ad769e22 P c182a7f66f6d49119a563266572c43b8: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/3d4e86b7e68f4df6a43689f1ad769e22
I20260808 13:03:29.520921 5089 log.cc:1196] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/3d4e86b7e68f4df6a43689f1ad769e22
I20260808 13:03:29.521381 5086 ts_tablet_manager.cc:1939] T 3d4e86b7e68f4df6a43689f1ad769e22 P 1019273dbea64a9ca91689ce685bc71a: Deleting consensus metadata
I20260808 13:03:29.521476 5087 ts_tablet_manager.cc:1939] T 3d4e86b7e68f4df6a43689f1ad769e22 P c182a7f66f6d49119a563266572c43b8: Deleting consensus metadata
I20260808 13:03:29.521476 5089 ts_tablet_manager.cc:1939] T 3d4e86b7e68f4df6a43689f1ad769e22 P fdde88d3d6dd4bf4a4bf7ca9ffbc4505: Deleting consensus metadata
Aug 08 13:03:29 dist-test-slave-t3q3 krb5kdc[4059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194207, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.124.129@KRBTEST.COM
I20260808 13:03:29.522655 4102 catalog_manager.cc:4841] TS fdde88d3d6dd4bf4a4bf7ca9ffbc4505 (127.1.124.129:37625): tablet 3d4e86b7e68f4df6a43689f1ad769e22 (table test-table [id=c0efcc4fc59e40c2b6909f3662ec1d69]) successfully deleted
I20260808 13:03:29.522657 4103 catalog_manager.cc:4841] TS 1019273dbea64a9ca91689ce685bc71a (127.1.124.130:41619): tablet 3d4e86b7e68f4df6a43689f1ad769e22 (table test-table [id=c0efcc4fc59e40c2b6909f3662ec1d69]) successfully deleted
I20260808 13:03:29.523206 4103 catalog_manager.cc:4841] TS c182a7f66f6d49119a563266572c43b8 (127.1.124.131:34681): tablet 3d4e86b7e68f4df6a43689f1ad769e22 (table test-table [id=c0efcc4fc59e40c2b6909f3662ec1d69]) successfully deleted
I20260808 13:03:29.528864 5093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:29.500515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35532 (local address 127.1.124.129:37625)
0808 13:03:29.511078 (+ 10563us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:29.511090 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:29.511124 (+ 34us) server_negotiation.cc:407] Connection header received
0808 13:03:29.511194 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:29.511200 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:29.511305 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:29.512350 (+ 1045us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:29.513622 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:29.514810 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:29.516735 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:29.517399 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:29.517533 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:29.523287 (+ 5754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:29.523310 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:29.523332 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:29.523382 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:29.526264 (+ 2882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:29.526928 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:29.526934 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:29.526937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:29.527067 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:29.527416 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:29.527419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:29.527421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:29.527760 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:29.527970 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:29.528316 (+ 346us) server_negotiation.cc:299] Negotiation successful
0808 13:03:29.528484 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9712,"thread_start_us":9519,"threads_started":1}
W20260808 13:03:29.529580 4433 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35532
W20260808 13:03:29.530244 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 4236: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:03:29.530471 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 4236
I20260808 13:03:29.566083 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 4445
Aug 08 13:03:29 dist-test-slave-t3q3 krb5kdc[4059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194207, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.124.130@KRBTEST.COM
I20260808 13:03:29.583194 5096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:29.566943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32800 (local address 127.1.124.130:41619)
0808 13:03:29.567516 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:29.567521 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:29.567538 (+ 17us) server_negotiation.cc:407] Connection header received
0808 13:03:29.567608 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:29.567612 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:29.567679 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:29.567811 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:29.569377 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:29.570259 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:29.571691 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:29.572294 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:29.572443 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:29.576978 (+ 4535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:29.576994 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:29.577012 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:29.577051 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:29.580236 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:29.580937 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:29.580947 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:29.580951 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:29.581159 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:29.581568 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:29.581573 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:29.581577 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:29.581941 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:29.582275 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:29.582594 (+ 319us) server_negotiation.cc:299] Negotiation successful
0808 13:03:29.582830 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":288,"threads_started":1}
W20260808 13:03:29.583891 4642 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:32800
W20260808 13:03:29.584589 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 4445: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:03:29.584786 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 4445
I20260808 13:03:29.617722 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 4653
Aug 08 13:03:29 dist-test-slave-t3q3 krb5kdc[4059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194207, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.1.124.131@KRBTEST.COM
I20260808 13:03:29.633750 5097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:29.618606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41568 (local address 127.1.124.131:34681)
0808 13:03:29.619192 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:29.619197 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:29.619215 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:03:29.619274 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:29.619278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:29.619346 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:29.619451 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:29.620878 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:29.621700 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:29.622949 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:29.623433 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:29.623550 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:29.627854 (+ 4304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:29.627870 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:29.627887 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:29.627931 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:29.631194 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:29.631794 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:29.631801 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:29.631804 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:29.631945 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:29.632347 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:29.632351 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:29.632353 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:29.632755 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:29.633017 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:29.633218 (+ 201us) server_negotiation.cc:299] Negotiation successful
0808 13:03:29.633394 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":319,"threads_started":1}
W20260808 13:03:29.634552 4849 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41568
W20260808 13:03:29.635284 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 4653: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:03:29.635490 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 4653
I20260808 13:03:29.671856 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 4080
I20260808 13:03:29.673225 4186 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:29.792025 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 4080
2026-08-08T13:03:29Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9151 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:03:29 dist-test-slave-t3q3 krb5kdc[5101](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:03:29 dist-test-slave-t3q3 krb5kdc[5101](info): set up 2 sockets
Aug 08 13:03:29 dist-test-slave-t3q3 krb5kdc[5101](info): commencing operation
krb5kdc: starting...
W20260808 13:03:31.115038 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.000s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:03:31 dist-test-slave-t3q3 krb5kdc[5101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:03:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:03:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:03:31.348783 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:41899
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:40911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:41899
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:31.686749 5118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:31.687074 5118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:31.687166 5118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:31.694834 5118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:03:31.694963 5118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:31.695015 5118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:31.695066 5118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:03:31.695103 5118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:03:31.705232 5118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:40911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:41899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:41899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:31.706527 5118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:31.707819 5118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:31.720158 5125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:31.720209 5126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:31.720357 5128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:31.721668 5118 server_base.cc:1034] running on GCE node
I20260808 13:03:31.722611 5118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:31.724244 5118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:31.725503 5118 hybrid_clock.cc:648] HybridClock initialized: now 1786194211725457 us; error 54 us; skew 500 ppm
I20260808 13:03:31.725898 5118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:31 dist-test-slave-t3q3 krb5kdc[5101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194211, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:31.734040 5118 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:03:31.735870 5118 webserver.cc:460] Webserver started at http://127.1.124.190:45265/ using document root <none> and password file <none>
I20260808 13:03:31.736689 5118 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:31.736861 5118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:31.737212 5118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:31.740438 5118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "233a2192ca9e49b0adbc30ef483739d5"
format_stamp: "Formatted at 2026-08-08 13:03:31 on dist-test-slave-t3q3"
server_key: "ab82c82ee60dbc816294a33bd8af1e84"
server_key_iv: "d21e410d273a6399eda085d78c349059"
server_key_version: "encryptionkey@0"
I20260808 13:03:31.741282 5118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "233a2192ca9e49b0adbc30ef483739d5"
format_stamp: "Formatted at 2026-08-08 13:03:31 on dist-test-slave-t3q3"
server_key: "ab82c82ee60dbc816294a33bd8af1e84"
server_key_iv: "d21e410d273a6399eda085d78c349059"
server_key_version: "encryptionkey@0"
I20260808 13:03:31.747009 5118 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 13:03:31.750674 5138 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:31.752127 5118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:03:31.752344 5118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "233a2192ca9e49b0adbc30ef483739d5"
format_stamp: "Formatted at 2026-08-08 13:03:31 on dist-test-slave-t3q3"
server_key: "ab82c82ee60dbc816294a33bd8af1e84"
server_key_iv: "d21e410d273a6399eda085d78c349059"
server_key_version: "encryptionkey@0"
I20260808 13:03:31.752569 5118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:31.778687 5118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:31.783577 5118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:31.783926 5118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:31.804116 5118 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:41899
I20260808 13:03:31.804214 5218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:41899 every 8 connection(s)
I20260808 13:03:31.805581 5118 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:03:31.809314 5220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:31.812474 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 5118
I20260808 13:03:31.812685 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:03:31.813509 1522 external_mini_cluster.cc:1428] Setting key 81a8e204cc2796ab48be8911f28534ae
I20260808 13:03:31.821698 5220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5: Bootstrap starting.
I20260808 13:03:31.825062 5220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:31.826193 5220 log.cc:824] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5: Log is configured to *not* fsync() on all Append() calls
Aug 08 13:03:31 dist-test-slave-t3q3 krb5kdc[5101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194211, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:31.829696 5220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5: No bootstrap required, opened a new log
I20260808 13:03:31.838366 5227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:31.815757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40508 (local address 127.1.124.190:41899)
0808 13:03:31.816538 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:31.816550 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:31.816590 (+ 40us) server_negotiation.cc:407] Connection header received
0808 13:03:31.818236 (+ 1646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:31.818259 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:31.818668 (+ 409us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:31.819091 (+ 423us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:31.820749 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:31.822181 (+ 1432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:31.823518 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:31.824321 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:31.824531 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:31.829445 (+ 4914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:31.829455 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:31.829472 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:31.829528 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:31.833271 (+ 3743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:31.834201 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:31.834209 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:31.834217 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:31.834412 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:31.834872 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:31.834878 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:31.834884 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:31.835578 (+ 694us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:31.835944 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:31.836595 (+ 651us) server_negotiation.cc:299] Negotiation successful
0808 13:03:31.837109 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":374,"threads_started":1}
I20260808 13:03:31.839371 5220 raft_consensus.cc:348] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233a2192ca9e49b0adbc30ef483739d5" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41899 } }
I20260808 13:03:31.839855 5220 raft_consensus.cc:374] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:31.840006 5220 raft_consensus.cc:729] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 233a2192ca9e49b0adbc30ef483739d5, State: Initialized, Role: FOLLOWER
I20260808 13:03:31.840828 5220 consensus_queue.cc:260] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233a2192ca9e49b0adbc30ef483739d5" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41899 } }
I20260808 13:03:31.841176 5220 raft_consensus.cc:388] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:03:31.841337 5220 raft_consensus.cc:482] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:03:31.841545 5220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:31.844192 5220 raft_consensus.cc:504] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233a2192ca9e49b0adbc30ef483739d5" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41899 } }
I20260808 13:03:31.844692 5220 leader_election.cc:302] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 233a2192ca9e49b0adbc30ef483739d5; no voters:
I20260808 13:03:31.845346 5220 leader_election.cc:288] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:03:31.845450 5231 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:31.846269 5231 raft_consensus.cc:686] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5 [term 1 LEADER]: Becoming Leader. State: Replica: 233a2192ca9e49b0adbc30ef483739d5, State: Running, Role: LEADER
I20260808 13:03:31.846930 5231 consensus_queue.cc:237] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233a2192ca9e49b0adbc30ef483739d5" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41899 } }
I20260808 13:03:31.847811 5220 sys_catalog.cc:564] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:03:31.850116 5232 sys_catalog.cc:455] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "233a2192ca9e49b0adbc30ef483739d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233a2192ca9e49b0adbc30ef483739d5" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41899 } } }
I20260808 13:03:31.850389 5233 sys_catalog.cc:455] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 233a2192ca9e49b0adbc30ef483739d5. Latest consensus state: current_term: 1 leader_uuid: "233a2192ca9e49b0adbc30ef483739d5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "233a2192ca9e49b0adbc30ef483739d5" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41899 } } }
I20260808 13:03:31.850602 5232 sys_catalog.cc:458] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5 [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:31.850678 5233 sys_catalog.cc:458] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5 [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:31.851471 5240 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:03:31.857712 5240 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:03:31.867254 5240 catalog_manager.cc:1347] Generated new cluster ID: bbfaad21046448d4a977a38ab29670ed
I20260808 13:03:31.867403 5240 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:03:31.877287 5240 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:03:31.879014 5240 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:03:31.888940 5240 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 233a2192ca9e49b0adbc30ef483739d5: Generated new TSK 0
I20260808 13:03:31.889801 5240 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.124.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
I20260808 13:03:31.952121 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:41899
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:40911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:32.309267 5264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:32.309562 5264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:32.309747 5264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:32.317257 5264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:32.317363 5264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:32.317579 5264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:03:32.327385 5264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:40911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:41899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:32.328819 5264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:32.330257 5264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:32.344611 5270 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:32.344724 5271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:32.344687 5273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:32.346077 5264 server_base.cc:1034] running on GCE node
I20260808 13:03:32.346817 5264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:32.347826 5264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:32.349067 5264 hybrid_clock.cc:648] HybridClock initialized: now 1786194212348987 us; error 74 us; skew 500 ppm
I20260808 13:03:32.349458 5264 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:32 dist-test-slave-t3q3 krb5kdc[5101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194212, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:32.352666 5264 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:03:32.354387 5264 webserver.cc:460] Webserver started at http://127.1.124.129:45031/ using document root <none> and password file <none>
I20260808 13:03:32.355099 5264 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:32.355244 5264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:32.355577 5264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:32.358865 5264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "68ab1fcb2a4741c18566e4e78415a98b"
format_stamp: "Formatted at 2026-08-08 13:03:32 on dist-test-slave-t3q3"
server_key: "ca7c605de17cddd297e947b751ee714a"
server_key_iv: "d66fce8615b67ebb2201bc5de80c0c1a"
server_key_version: "encryptionkey@0"
I20260808 13:03:32.359689 5264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "68ab1fcb2a4741c18566e4e78415a98b"
format_stamp: "Formatted at 2026-08-08 13:03:32 on dist-test-slave-t3q3"
server_key: "ca7c605de17cddd297e947b751ee714a"
server_key_iv: "d66fce8615b67ebb2201bc5de80c0c1a"
server_key_version: "encryptionkey@0"
I20260808 13:03:32.365160 5264 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 13:03:32.368643 5281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:32.369961 5264 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 13:03:32.370170 5264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "68ab1fcb2a4741c18566e4e78415a98b"
format_stamp: "Formatted at 2026-08-08 13:03:32 on dist-test-slave-t3q3"
server_key: "ca7c605de17cddd297e947b751ee714a"
server_key_iv: "d66fce8615b67ebb2201bc5de80c0c1a"
server_key_version: "encryptionkey@0"
I20260808 13:03:32.370376 5264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:32.400333 5264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:32.405246 5264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:32.405704 5264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:32.406810 5264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:03:32.408363 5264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:32.408448 5264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:32.408615 5264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:32.408690 5264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:32.445950 5264 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:42871
I20260808 13:03:32.445987 5446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:42871 every 8 connection(s)
I20260808 13:03:32.447537 5264 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:03:32.451563 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 5264
I20260808 13:03:32.451777 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:03:32.452696 1522 external_mini_cluster.cc:1428] Setting key e0564a77cb56f7f8bdc36d9d7bc45b60
Aug 08 13:03:32 dist-test-slave-t3q3 krb5kdc[5101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194212, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:32.472774 5452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:32.450477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:41893 (local address 127.1.124.190:41899)
0808 13:03:32.450964 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:32.450968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:32.452096 (+ 1128us) server_negotiation.cc:407] Connection header received
0808 13:03:32.454859 (+ 2763us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:32.454864 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:32.454938 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:32.455061 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:32.457273 (+ 2212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:32.458061 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:32.460681 (+ 2620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:32.461130 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:32.461243 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:32.465507 (+ 4264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:32.465517 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:32.465525 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:32.465573 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:32.469298 (+ 3725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:32.470088 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:32.470097 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:32.470102 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:32.470258 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:32.470668 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:32.470675 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:32.470680 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:32.471060 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:32.471228 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:32.471961 (+ 733us) server_negotiation.cc:299] Negotiation successful
0808 13:03:32.472245 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":269,"threads_started":1}
I20260808 13:03:32.473520 5453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:32.450933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:41899 (local address 127.1.124.129:41893)
0808 13:03:32.451858 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:32.451915 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:32.454543 (+ 2628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:32.455348 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:32.455362 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:32.456213 (+ 851us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:32.457081 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:32.457111 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:32.458237 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:32.458244 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:32.460494 (+ 2250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:32.460520 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:32.461370 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:32.461380 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:32.461724 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:32.462543 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:32.462578 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:32.465249 (+ 2671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:32.469460 (+ 4211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:32.469468 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:32.469482 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:32.469938 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:32.470367 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:32.470370 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:32.470372 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:32.470543 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:32.471196 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:32.471202 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:32.471530 (+ 328us) client_negotiation.cc:769] Sending connection context
0808 13:03:32.471782 (+ 252us) client_negotiation.cc:240] Negotiation successful
0808 13:03:32.472261 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":334,"threads_started":1}
I20260808 13:03:32.475234 5449 heartbeater.cc:347] Connected to a master server at 127.1.124.190:41899
I20260808 13:03:32.475574 5449 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:32.476379 5449 heartbeater.cc:511] Master 127.1.124.190:41899 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.124.130@KRBTEST.COM; defaulting to no policy
I20260808 13:03:32.478543 5160 ts_manager.cc:194] Registered new tserver with Master: 68ab1fcb2a4741c18566e4e78415a98b (127.1.124.129:42871)
I20260808 13:03:32.480361 5160 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:41893
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
I20260808 13:03:32.508132 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:41899
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:40911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:32.846808 5460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:32.847112 5460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:32.847283 5460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:32.854918 5460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:32.855034 5460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:32.855230 5460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:03:32.865561 5460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:40911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:41899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:32.866930 5460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:32.868333 5460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:32.882200 5474 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:32.882110 5468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:32.882551 5470 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:32.883324 5460 server_base.cc:1034] running on GCE node
I20260808 13:03:32.883908 5460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:32.884925 5460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:32.886135 5460 hybrid_clock.cc:648] HybridClock initialized: now 1786194212886089 us; error 29 us; skew 500 ppm
I20260808 13:03:32.886502 5460 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:32 dist-test-slave-t3q3 krb5kdc[5101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194212, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:32.890159 5460 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:03:32.891753 5460 webserver.cc:460] Webserver started at http://127.1.124.130:34685/ using document root <none> and password file <none>
I20260808 13:03:32.892710 5460 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:32.892886 5460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:32.893257 5460 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:32.896466 5460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "8979646a36de4dde8a7d64d1e45f7fe2"
format_stamp: "Formatted at 2026-08-08 13:03:32 on dist-test-slave-t3q3"
server_key: "83aac04e7310a40e3ab8e20b55beec43"
server_key_iv: "a278145dbe93e2597c08e8b129b9723a"
server_key_version: "encryptionkey@0"
I20260808 13:03:32.897315 5460 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "8979646a36de4dde8a7d64d1e45f7fe2"
format_stamp: "Formatted at 2026-08-08 13:03:32 on dist-test-slave-t3q3"
server_key: "83aac04e7310a40e3ab8e20b55beec43"
server_key_iv: "a278145dbe93e2597c08e8b129b9723a"
server_key_version: "encryptionkey@0"
I20260808 13:03:32.902773 5460 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:03:32.906418 5486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:32.907992 5460 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 13:03:32.908255 5460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "8979646a36de4dde8a7d64d1e45f7fe2"
format_stamp: "Formatted at 2026-08-08 13:03:32 on dist-test-slave-t3q3"
server_key: "83aac04e7310a40e3ab8e20b55beec43"
server_key_iv: "a278145dbe93e2597c08e8b129b9723a"
server_key_version: "encryptionkey@0"
I20260808 13:03:32.908490 5460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:32.924739 5460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:32.929551 5460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:32.929951 5460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:32.930994 5460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:03:32.932539 5460 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:32.932672 5460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:32.932829 5460 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:32.932916 5460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:32.970281 5460 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:33985
I20260808 13:03:32.970317 5659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:33985 every 8 connection(s)
I20260808 13:03:32.971760 5460 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:03:32.981132 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 5460
I20260808 13:03:32.981333 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:03:32.981777 1522 external_mini_cluster.cc:1428] Setting key a980ea64593a8e241092c8217f94c669
Aug 08 13:03:32 dist-test-slave-t3q3 krb5kdc[5101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194212, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:32.993348 5664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:32.974990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:51747 (local address 127.1.124.190:41899)
0808 13:03:32.975468 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:32.975472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:32.976594 (+ 1122us) server_negotiation.cc:407] Connection header received
0808 13:03:32.978475 (+ 1881us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:32.978480 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:32.978572 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:32.978710 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:32.980324 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:32.981074 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:32.982990 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:32.983586 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:32.983722 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:32.987512 (+ 3790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:32.987517 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:32.987520 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:32.987550 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:32.990174 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:32.990922 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:32.990927 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:32.990929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:32.991061 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:32.991537 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:32.991560 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:32.991562 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:32.991847 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:32.992005 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:32.992728 (+ 723us) server_negotiation.cc:299] Negotiation successful
0808 13:03:32.992953 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":257,"threads_started":1}
I20260808 13:03:32.994354 5665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:32.975636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:41899 (local address 127.1.124.130:51747)
0808 13:03:32.976424 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:32.976473 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:32.978195 (+ 1722us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:32.978964 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:32.978972 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:32.979495 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:32.980151 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:32.980172 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:32.981264 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:32.981288 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:32.982839 (+ 1551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:32.982851 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:32.983833 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:32.983841 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:32.984139 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:32.984850 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:32.984872 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:32.987276 (+ 2404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:32.990328 (+ 3052us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:32.990336 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:32.990353 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:32.990782 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:32.991185 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:32.991190 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:32.991194 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:32.991395 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:32.991980 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:32.991988 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:32.992364 (+ 376us) client_negotiation.cc:769] Sending connection context
0808 13:03:32.992633 (+ 269us) client_negotiation.cc:240] Negotiation successful
0808 13:03:32.993197 (+ 564us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":287,"threads_started":1}
I20260808 13:03:32.995874 5662 heartbeater.cc:347] Connected to a master server at 127.1.124.190:41899
I20260808 13:03:32.996235 5662 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:32.997120 5662 heartbeater.cc:511] Master 127.1.124.190:41899 requested a full tablet report, sending...
I20260808 13:03:32.998869 5160 ts_manager.cc:194] Registered new tserver with Master: 8979646a36de4dde8a7d64d1e45f7fe2 (127.1.124.130:33985)
I20260808 13:03:33.000206 5160 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:51747
WARNING: no policy specified for kudu/127.1.124.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
I20260808 13:03:33.033185 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:41899
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:40911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:33.387300 5672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:33.387609 5672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:33.387705 5672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:33.395421 5672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:33.395556 5672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:33.395675 5672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:03:33.406314 5672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:40911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:41899
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:33.407646 5672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:33.409003 5672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:33.422820 5680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:33.422780 5685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:33.422729 5682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:33.423730 5672 server_base.cc:1034] running on GCE node
I20260808 13:03:33.424587 5672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:33.425568 5672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:33.426779 5672 hybrid_clock.cc:648] HybridClock initialized: now 1786194213426719 us; error 60 us; skew 500 ppm
I20260808 13:03:33.427110 5672 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:33 dist-test-slave-t3q3 krb5kdc[5101](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194213, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:33.430361 5672 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:03:33.432197 5672 webserver.cc:460] Webserver started at http://127.1.124.131:36831/ using document root <none> and password file <none>
I20260808 13:03:33.432932 5672 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:33.433097 5672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:33.433418 5672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:33.436486 5672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "0c8b846f81d4430c9076573092cfc7de"
format_stamp: "Formatted at 2026-08-08 13:03:33 on dist-test-slave-t3q3"
server_key: "e430b3980b8c8f438769ca349cb6a7d8"
server_key_iv: "fbc682a308a074dc256f9f3165754148"
server_key_version: "encryptionkey@0"
I20260808 13:03:33.437310 5672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "0c8b846f81d4430c9076573092cfc7de"
format_stamp: "Formatted at 2026-08-08 13:03:33 on dist-test-slave-t3q3"
server_key: "e430b3980b8c8f438769ca349cb6a7d8"
server_key_iv: "fbc682a308a074dc256f9f3165754148"
server_key_version: "encryptionkey@0"
I20260808 13:03:33.442689 5672 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:03:33.446523 5698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:33.448051 5672 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:03:33.448266 5672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "0c8b846f81d4430c9076573092cfc7de"
format_stamp: "Formatted at 2026-08-08 13:03:33 on dist-test-slave-t3q3"
server_key: "e430b3980b8c8f438769ca349cb6a7d8"
server_key_iv: "fbc682a308a074dc256f9f3165754148"
server_key_version: "encryptionkey@0"
I20260808 13:03:33.448454 5672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:33.465351 5672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:33.469429 5672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:33.469728 5672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:33.470669 5672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:03:33.472035 5672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:33.472210 5672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260808 13:03:33.472368 5672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:33.472443 5672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:33.483573 5449 heartbeater.cc:503] Master 127.1.124.190:41899 was elected leader, sending a full tablet report...
I20260808 13:03:33.507503 5672 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:37215
I20260808 13:03:33.507550 5857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:37215 every 8 connection(s)
I20260808 13:03:33.508872 5672 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:03:33.516609 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 5672
I20260808 13:03:33.516839 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:03:33.517346 1522 external_mini_cluster.cc:1428] Setting key ce1a99b221a6a569ad43e01eb69c8df2
Aug 08 13:03:33 dist-test-slave-t3q3 krb5kdc[5101](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194213, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:33.529747 5860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:33.511591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:51841 (local address 127.1.124.190:41899)
0808 13:03:33.512160 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:33.512164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:33.513274 (+ 1110us) server_negotiation.cc:407] Connection header received
0808 13:03:33.515076 (+ 1802us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:33.515080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:33.515147 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:33.515249 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:33.517068 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:33.518017 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:33.520134 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:33.520693 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:33.520792 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:33.524230 (+ 3438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:33.524234 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:33.524238 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:33.524270 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:33.526788 (+ 2518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:33.527499 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:33.527502 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:33.527505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:33.527621 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:33.528001 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:33.528004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:33.528006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:33.528338 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:33.528492 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:33.529287 (+ 795us) server_negotiation.cc:299] Negotiation successful
0808 13:03:33.529459 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":286,"threads_started":1}
I20260808 13:03:33.530901 5862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:33.512262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:41899 (local address 127.1.124.131:51841)
0808 13:03:33.513063 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:33.513106 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:33.514814 (+ 1708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:33.515459 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:33.515467 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:33.515969 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:33.516875 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:33.516907 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:33.518225 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:33.518231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:33.519968 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:33.519981 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:33.520894 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:33.520899 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:33.521093 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:33.521749 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:33.521771 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:33.523945 (+ 2174us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:33.526924 (+ 2979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:33.526932 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:33.526945 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:33.527393 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:33.527717 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:33.527720 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:33.527722 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:33.527907 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:33.528506 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:33.528512 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:33.528905 (+ 393us) client_negotiation.cc:769] Sending connection context
0808 13:03:33.529196 (+ 291us) client_negotiation.cc:240] Negotiation successful
0808 13:03:33.529609 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":465,"thread_start_us":264,"threads_started":1}
I20260808 13:03:33.532442 5858 heartbeater.cc:347] Connected to a master server at 127.1.124.190:41899
I20260808 13:03:33.532809 5858 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:33.533582 5858 heartbeater.cc:511] Master 127.1.124.190:41899 requested a full tablet report, sending...
I20260808 13:03:33.535086 5159 ts_manager.cc:194] Registered new tserver with Master: 0c8b846f81d4430c9076573092cfc7de (127.1.124.131:37215)
I20260808 13:03:33.535944 5159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:51841
I20260808 13:03:33.545639 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 13:03:33.585014 5860 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:03:33.582531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40512 (local address 127.1.124.190:41899)
0808 13:03:33.582729 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:33.582734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:33.583197 (+ 463us) server_negotiation.cc:407] Connection header received
0808 13:03:33.583344 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:33.583348 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:33.583416 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:33.583508 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:33.584817 (+ 1309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40512: BlockingRecv error: recv got EOF from 127.0.0.1:40512 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 13:03:33.588491 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 5264
W20260808 13:03:33.591301 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 5264: Not authorized: Client connection negotiation failed: client connection to 127.1.124.129:42871: server requires authentication, but client does not have Kerberos credentials available
I20260808 13:03:33.591535 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 5264
I20260808 13:03:33.616605 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 5460
W20260808 13:03:33.619510 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 5460: Not authorized: Client connection negotiation failed: client connection to 127.1.124.130:33985: server requires authentication, but client does not have Kerberos credentials available
I20260808 13:03:33.619692 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 5460
W20260808 13:03:33.619822 5876 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:03:33.617620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46746 (local address 127.1.124.130:33985)
0808 13:03:33.618183 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:33.618190 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:33.618216 (+ 26us) server_negotiation.cc:407] Connection header received
0808 13:03:33.618280 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:33.618286 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:33.618499 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:33.618655 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:33.619504 (+ 849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46746: BlockingRecv error: recv got EOF from 127.0.0.1:46746 (error 108)
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":294,"threads_started":1}
I20260808 13:03:33.643097 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 5672
W20260808 13:03:33.646231 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 5672: Not authorized: Client connection negotiation failed: client connection to 127.1.124.131:37215: server requires authentication, but client does not have Kerberos credentials available
I20260808 13:03:33.646437 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 5672
I20260808 13:03:33.673508 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 5118
I20260808 13:03:33.674865 5210 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:33.785473 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 5118
2026-08-08T13:03:33Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3991 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:03:33 dist-test-slave-t3q3 krb5kdc[5886](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:03:33 dist-test-slave-t3q3 krb5kdc[5886](info): set up 2 sockets
Aug 08 13:03:33 dist-test-slave-t3q3 krb5kdc[5886](info): commencing operation
krb5kdc: starting...
W20260808 13:03:35.106988 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.001s sys 0.025s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:03:35 dist-test-slave-t3q3 krb5kdc[5886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:03:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:03:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:03:35.339497 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:43845
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:38619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:43845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:35.695425 5906 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:35.695744 5906 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:35.695901 5906 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:35.703405 5906 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:03:35.703516 5906 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:35.703619 5906 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:35.703696 5906 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:03:35.703773 5906 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:03:35.714010 5906 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:38619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:43845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:43845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:35.715390 5906 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:35.716751 5906 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:35.728817 5918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:35.728775 5916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:35.728874 5915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:35.729830 5906 server_base.cc:1034] running on GCE node
I20260808 13:03:35.730623 5906 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:35.732120 5906 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:35.733366 5906 hybrid_clock.cc:648] HybridClock initialized: now 1786194215733322 us; error 38 us; skew 500 ppm
I20260808 13:03:35.733745 5906 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:35 dist-test-slave-t3q3 krb5kdc[5886](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194215, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:35.736948 5906 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:03:35.738421 5906 webserver.cc:460] Webserver started at http://127.1.124.190:45241/ using document root <none> and password file <none>
I20260808 13:03:35.739122 5906 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:35.739279 5906 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:35.739678 5906 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:35.742769 5906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "59ed1a19d8f84366abd338ef7ef6a6dc"
format_stamp: "Formatted at 2026-08-08 13:03:35 on dist-test-slave-t3q3"
server_key: "c248f5710613977a4dd7e8106d034339"
server_key_iv: "5eb0fdc5d9f2bfba57f7545e01a29302"
server_key_version: "encryptionkey@0"
I20260808 13:03:35.743583 5906 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "59ed1a19d8f84366abd338ef7ef6a6dc"
format_stamp: "Formatted at 2026-08-08 13:03:35 on dist-test-slave-t3q3"
server_key: "c248f5710613977a4dd7e8106d034339"
server_key_iv: "5eb0fdc5d9f2bfba57f7545e01a29302"
server_key_version: "encryptionkey@0"
I20260808 13:03:35.749433 5906 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260808 13:03:35.752928 5926 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:35.754278 5906 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:03:35.754469 5906 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "59ed1a19d8f84366abd338ef7ef6a6dc"
format_stamp: "Formatted at 2026-08-08 13:03:35 on dist-test-slave-t3q3"
server_key: "c248f5710613977a4dd7e8106d034339"
server_key_iv: "5eb0fdc5d9f2bfba57f7545e01a29302"
server_key_version: "encryptionkey@0"
I20260808 13:03:35.754683 5906 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:35.767283 5906 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:35.771621 5906 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:35.771930 5906 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:35.791220 5906 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:43845
I20260808 13:03:35.791260 6001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:43845 every 8 connection(s)
I20260808 13:03:35.792716 5906 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:03:35.796586 6004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:35.803186 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 5906
I20260808 13:03:35.803442 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:03:35.804109 1522 external_mini_cluster.cc:1428] Setting key e862df5b2c39bd5067fdc23a47296913
I20260808 13:03:35.807250 6004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc: Bootstrap starting.
I20260808 13:03:35.810871 6004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:35.812141 6004 log.cc:824] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:35.814888 6004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc: No bootstrap required, opened a new log
Aug 08 13:03:35 dist-test-slave-t3q3 krb5kdc[5886](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194215, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:35.822420 6004 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ed1a19d8f84366abd338ef7ef6a6dc" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43845 } }
I20260808 13:03:35.822820 6004 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:35.822934 6004 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59ed1a19d8f84366abd338ef7ef6a6dc, State: Initialized, Role: FOLLOWER
I20260808 13:03:35.823621 6004 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ed1a19d8f84366abd338ef7ef6a6dc" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43845 } }
I20260808 13:03:35.823937 6004 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:03:35.824142 6004 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:03:35.824352 6004 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:35.826843 6008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:35.806051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43784 (local address 127.1.124.190:43845)
0808 13:03:35.806866 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:35.806877 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:35.806921 (+ 44us) server_negotiation.cc:407] Connection header received
0808 13:03:35.808785 (+ 1864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:35.808811 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:35.809191 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:35.809502 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:35.811490 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:35.812930 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:35.814236 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:35.814902 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:35.815098 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:35.819000 (+ 3902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:35.819007 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:35.819022 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:35.819065 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:35.822273 (+ 3208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:35.823077 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:35.823093 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:35.823099 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:35.823320 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:35.823838 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:35.823841 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:35.823843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:35.824619 (+ 776us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:35.824848 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:35.825365 (+ 517us) server_negotiation.cc:299] Negotiation successful
0808 13:03:35.825780 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":392,"threads_started":1}
I20260808 13:03:35.827287 6004 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ed1a19d8f84366abd338ef7ef6a6dc" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43845 } }
I20260808 13:03:35.827812 6004 leader_election.cc:302] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59ed1a19d8f84366abd338ef7ef6a6dc; no voters:
I20260808 13:03:35.828574 6004 leader_election.cc:288] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:03:35.828713 6010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:35.830529 6010 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc [term 1 LEADER]: Becoming Leader. State: Replica: 59ed1a19d8f84366abd338ef7ef6a6dc, State: Running, Role: LEADER
I20260808 13:03:35.831122 6004 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:03:35.831192 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:03:35.831157 6010 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ed1a19d8f84366abd338ef7ef6a6dc" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43845 } }
I20260808 13:03:35.842852 6012 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59ed1a19d8f84366abd338ef7ef6a6dc. Latest consensus state: current_term: 1 leader_uuid: "59ed1a19d8f84366abd338ef7ef6a6dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ed1a19d8f84366abd338ef7ef6a6dc" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43845 } } }
I20260808 13:03:35.843254 6012 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:35.853350 6011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59ed1a19d8f84366abd338ef7ef6a6dc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ed1a19d8f84366abd338ef7ef6a6dc" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43845 } } }
I20260808 13:03:35.853765 6011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:35.854765 6026 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:03:35.859663 6026 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:03:35.868021 6026 catalog_manager.cc:1347] Generated new cluster ID: e853833284e84bbb8740f6328f0e2d0a
I20260808 13:03:35.868193 6026 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:03:35.890605 6026 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:03:35.892094 6008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:35.882436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43786 (local address 127.1.124.190:43845)
0808 13:03:35.882626 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:35.882633 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:35.883258 (+ 625us) server_negotiation.cc:407] Connection header received
0808 13:03:35.883454 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:35.883460 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:35.883541 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:35.883653 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:35.885018 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:35.886174 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:35.888105 (+ 1931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:35.888599 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:35.888764 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:35.890108 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:35.890115 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:35.890120 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:03:35.891133 (+ 1013us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:35.891227 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:03:35.891275 (+ 48us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:35.891449 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:35.891565 (+ 116us) server_negotiation.cc:299] Negotiation successful
0808 13:03:35.891666 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 13:03:35.892437 6026 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:03:35.904685 6026 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59ed1a19d8f84366abd338ef7ef6a6dc: Generated new TSK 0
I20260808 13:03:35.905829 6026 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:03:35.918895 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 5906
I20260808 13:03:35.920583 5994 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:36.012168 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 5906
2026-08-08T13:03:36Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2216 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:03:36 dist-test-slave-t3q3 krb5kdc[6056](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:03:36 dist-test-slave-t3q3 krb5kdc[6056](info): set up 2 sockets
Aug 08 13:03:36 dist-test-slave-t3q3 krb5kdc[6056](info): commencing operation
krb5kdc: starting...
W20260808 13:03:37.318364 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.263s user 0.001s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:03:37 dist-test-slave-t3q3 krb5kdc[6056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:03:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:03:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:03:37.564853 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:39513
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:39513
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:37.905450 6077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:37.905778 6077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:37.905987 6077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:37.913431 6077 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:03:37.913560 6077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:37.913671 6077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:37.913774 6077 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:03:37.913839 6077 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:03:37.923945 6077 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:39513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:39513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:37.925439 6077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:37.926780 6077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:37.938995 6086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:37.939028 6089 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:37.939286 6087 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:37.940001 6077 server_base.cc:1034] running on GCE node
I20260808 13:03:37.940940 6077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:37.942479 6077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:37.943724 6077 hybrid_clock.cc:648] HybridClock initialized: now 1786194217943674 us; error 50 us; skew 500 ppm
I20260808 13:03:37.944146 6077 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:37 dist-test-slave-t3q3 krb5kdc[6056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194217, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:37.950697 6077 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:03:37.952348 6077 webserver.cc:460] Webserver started at http://127.1.124.190:35253/ using document root <none> and password file <none>
I20260808 13:03:37.953051 6077 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:37.953161 6077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:37.953423 6077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:37.956640 6077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "df41e20fc6344b7c8520c1922b5b1823"
format_stamp: "Formatted at 2026-08-08 13:03:37 on dist-test-slave-t3q3"
server_key: "df845564bce5b2c8380f3543f9a28dd7"
server_key_iv: "24e3c8df78bc63e9b31954fbf613e4fc"
server_key_version: "encryptionkey@0"
I20260808 13:03:37.957440 6077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "df41e20fc6344b7c8520c1922b5b1823"
format_stamp: "Formatted at 2026-08-08 13:03:37 on dist-test-slave-t3q3"
server_key: "df845564bce5b2c8380f3543f9a28dd7"
server_key_iv: "24e3c8df78bc63e9b31954fbf613e4fc"
server_key_version: "encryptionkey@0"
I20260808 13:03:37.962937 6077 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 13:03:37.966562 6097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:37.968170 6077 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 13:03:37.968379 6077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "df41e20fc6344b7c8520c1922b5b1823"
format_stamp: "Formatted at 2026-08-08 13:03:37 on dist-test-slave-t3q3"
server_key: "df845564bce5b2c8380f3543f9a28dd7"
server_key_iv: "24e3c8df78bc63e9b31954fbf613e4fc"
server_key_version: "encryptionkey@0"
I20260808 13:03:37.968600 6077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:37.991894 6077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:37.996587 6077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:37.996920 6077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:38.016414 6077 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:39513
I20260808 13:03:38.016492 6183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:39513 every 8 connection(s)
I20260808 13:03:38.017817 6077 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:03:38.018795 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 6077
I20260808 13:03:38.019081 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:03:38.019764 1522 external_mini_cluster.cc:1428] Setting key f5ae7f4e96cf98e212251f69d388a7fd
I20260808 13:03:38.022522 6184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 13:03:38 dist-test-slave-t3q3 krb5kdc[6056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194217, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:38.035996 6184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823: Bootstrap starting.
I20260808 13:03:38.039268 6184 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:38.040390 6184 log.cc:824] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:38.043473 6184 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823: No bootstrap required, opened a new log
I20260808 13:03:38.043502 6190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:38.021768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33596 (local address 127.1.124.190:39513)
0808 13:03:38.022542 (+ 774us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:38.022553 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:38.022623 (+ 70us) server_negotiation.cc:407] Connection header received
0808 13:03:38.024334 (+ 1711us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:38.024356 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:38.024737 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:38.025096 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:38.026911 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:38.028414 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:38.030215 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:38.030918 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:38.031046 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:38.035947 (+ 4901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:38.035955 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:38.035972 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:38.036012 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:38.039227 (+ 3215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:38.040006 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:38.040013 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:38.040020 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:38.040267 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:38.040687 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:38.040692 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:38.040697 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:38.041316 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:38.041672 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:38.042116 (+ 444us) server_negotiation.cc:299] Negotiation successful
0808 13:03:38.042547 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":351,"threads_started":1}
I20260808 13:03:38.051682 6184 raft_consensus.cc:348] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df41e20fc6344b7c8520c1922b5b1823" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39513 } }
I20260808 13:03:38.052218 6184 raft_consensus.cc:374] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:38.052345 6184 raft_consensus.cc:729] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df41e20fc6344b7c8520c1922b5b1823, State: Initialized, Role: FOLLOWER
I20260808 13:03:38.052939 6184 consensus_queue.cc:260] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df41e20fc6344b7c8520c1922b5b1823" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39513 } }
I20260808 13:03:38.053217 6184 raft_consensus.cc:388] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:03:38.053366 6184 raft_consensus.cc:482] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:03:38.053560 6184 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:38.055898 6184 raft_consensus.cc:504] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df41e20fc6344b7c8520c1922b5b1823" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39513 } }
I20260808 13:03:38.056473 6184 leader_election.cc:302] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df41e20fc6344b7c8520c1922b5b1823; no voters:
I20260808 13:03:38.057309 6184 leader_election.cc:288] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:03:38.057452 6192 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:38.058246 6192 raft_consensus.cc:686] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823 [term 1 LEADER]: Becoming Leader. State: Replica: df41e20fc6344b7c8520c1922b5b1823, State: Running, Role: LEADER
I20260808 13:03:38.058815 6192 consensus_queue.cc:237] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df41e20fc6344b7c8520c1922b5b1823" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39513 } }
I20260808 13:03:38.059554 6184 sys_catalog.cc:564] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:03:38.061496 6194 sys_catalog.cc:455] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df41e20fc6344b7c8520c1922b5b1823" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df41e20fc6344b7c8520c1922b5b1823" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39513 } } }
I20260808 13:03:38.061894 6194 sys_catalog.cc:458] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823 [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:38.062202 6195 sys_catalog.cc:455] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df41e20fc6344b7c8520c1922b5b1823. Latest consensus state: current_term: 1 leader_uuid: "df41e20fc6344b7c8520c1922b5b1823" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df41e20fc6344b7c8520c1922b5b1823" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39513 } } }
I20260808 13:03:38.062620 6195 sys_catalog.cc:458] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823 [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:38.064729 6199 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:03:38.069922 6199 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:03:38.079644 6199 catalog_manager.cc:1347] Generated new cluster ID: 18b00005f5134e2495db718966593d7f
I20260808 13:03:38.079769 6199 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:03:38.103613 6199 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:03:38.105587 6199 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:03:38.120916 6199 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df41e20fc6344b7c8520c1922b5b1823: Generated new TSK 0
I20260808 13:03:38.121990 6199 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.124.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
I20260808 13:03:38.184736 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:39513
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:38.559574 6226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:38.559856 6226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:38.559952 6226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:38.567737 6226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:38.567858 6226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:38.567975 6226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:03:38.578462 6226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:39513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:38.579838 6226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:38.581173 6226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:38.594581 6236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:38.594583 6237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:38.594825 6240 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:38.595752 6226 server_base.cc:1034] running on GCE node
I20260808 13:03:38.596505 6226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:38.597514 6226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:38.598721 6226 hybrid_clock.cc:648] HybridClock initialized: now 1786194218598670 us; error 43 us; skew 500 ppm
I20260808 13:03:38.599102 6226 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:38 dist-test-slave-t3q3 krb5kdc[6056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194218, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:38.602550 6226 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:03:38.604133 6226 webserver.cc:460] Webserver started at http://127.1.124.129:45935/ using document root <none> and password file <none>
I20260808 13:03:38.604820 6226 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:38.604931 6226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:38.605188 6226 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:38.608204 6226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "6ce31b17a8ee44198c3be1378af71ef3"
format_stamp: "Formatted at 2026-08-08 13:03:38 on dist-test-slave-t3q3"
server_key: "e98e4c92481a4d0e82f1e6c5eab0885d"
server_key_iv: "0aab9ed490518354c5b15b53bdfcb56b"
server_key_version: "encryptionkey@0"
I20260808 13:03:38.608978 6226 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "6ce31b17a8ee44198c3be1378af71ef3"
format_stamp: "Formatted at 2026-08-08 13:03:38 on dist-test-slave-t3q3"
server_key: "e98e4c92481a4d0e82f1e6c5eab0885d"
server_key_iv: "0aab9ed490518354c5b15b53bdfcb56b"
server_key_version: "encryptionkey@0"
I20260808 13:03:38.614151 6226 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260808 13:03:38.617709 6248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:38.619295 6226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:03:38.619488 6226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "6ce31b17a8ee44198c3be1378af71ef3"
format_stamp: "Formatted at 2026-08-08 13:03:38 on dist-test-slave-t3q3"
server_key: "e98e4c92481a4d0e82f1e6c5eab0885d"
server_key_iv: "0aab9ed490518354c5b15b53bdfcb56b"
server_key_version: "encryptionkey@0"
I20260808 13:03:38.619678 6226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:38.635435 6226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:38.639771 6226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:38.640166 6226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:38.641208 6226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:03:38.642607 6226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:38.642733 6226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:38.642889 6226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:38.642964 6226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:38.678323 6226 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:38047
I20260808 13:03:38.678363 6424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:38047 every 8 connection(s)
I20260808 13:03:38.679665 6226 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:03:38.680997 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 6226
I20260808 13:03:38.681252 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:03:38.682116 1522 external_mini_cluster.cc:1428] Setting key c3a466b862306724a8dbccefc09aa277
Aug 08 13:03:38 dist-test-slave-t3q3 krb5kdc[6056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194218, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:38.704713 6427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:38.683020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:48153 (local address 127.1.124.190:39513)
0808 13:03:38.683472 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:38.683477 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:38.684319 (+ 842us) server_negotiation.cc:407] Connection header received
0808 13:03:38.687038 (+ 2719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:38.687042 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:38.687108 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:38.687205 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:38.689531 (+ 2326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:38.690364 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:38.692980 (+ 2616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:38.693468 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:38.693605 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:38.697567 (+ 3962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:38.697577 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:38.697584 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:38.697631 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:38.701238 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:38.702008 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:38.702016 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:38.702022 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:38.702179 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:38.702616 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:38.702624 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:38.702630 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:38.703044 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:38.703206 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:38.703949 (+ 743us) server_negotiation.cc:299] Negotiation successful
0808 13:03:38.704249 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":338,"thread_start_us":230,"threads_started":1}
I20260808 13:03:38.705479 6428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:38.683162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39513 (local address 127.1.124.129:48153)
0808 13:03:38.684126 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:38.684196 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:38.686714 (+ 2518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:38.687515 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:38.687529 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:38.688398 (+ 869us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:38.689293 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:38.689323 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:38.690565 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:38.690574 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:38.692791 (+ 2217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:38.692810 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:38.693755 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:38.693764 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:38.694121 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:38.694927 (+ 806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:38.694962 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:38.697292 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:38.701380 (+ 4088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:38.701387 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:38.701401 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:38.701859 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:38.702288 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:38.702291 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:38.702294 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:38.702476 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:38.703169 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:38.703175 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:38.703513 (+ 338us) client_negotiation.cc:769] Sending connection context
0808 13:03:38.703755 (+ 242us) client_negotiation.cc:240] Negotiation successful
0808 13:03:38.704323 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":350,"threads_started":1}
I20260808 13:03:38.707209 6425 heartbeater.cc:347] Connected to a master server at 127.1.124.190:39513
I20260808 13:03:38.707583 6425 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.1.124.130@KRBTEST.COM; defaulting to no policy
I20260808 13:03:38.708623 6425 heartbeater.cc:511] Master 127.1.124.190:39513 requested a full tablet report, sending...
I20260808 13:03:38.710917 6127 ts_manager.cc:194] Registered new tserver with Master: 6ce31b17a8ee44198c3be1378af71ef3 (127.1.124.129:38047)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.130@KRBTEST.COM" created.
I20260808 13:03:38.713053 6127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:48153
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
I20260808 13:03:38.739275 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:39513
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:39.084405 6431 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:39.084690 6431 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:39.084789 6431 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:39.091917 6431 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:39.092038 6431 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:39.092248 6431 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:03:39.102116 6431 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:39513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:39.103472 6431 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:39.104794 6431 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:39.118551 6446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:39.118687 6453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:39.118677 6448 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:39.119395 6431 server_base.cc:1034] running on GCE node
I20260808 13:03:39.120023 6431 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:39.121183 6431 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:39.122426 6431 hybrid_clock.cc:648] HybridClock initialized: now 1786194219122343 us; error 90 us; skew 500 ppm
I20260808 13:03:39.122715 6431 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:39 dist-test-slave-t3q3 krb5kdc[6056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194219, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:39.126121 6431 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:03:39.127699 6431 webserver.cc:460] Webserver started at http://127.1.124.130:38523/ using document root <none> and password file <none>
I20260808 13:03:39.128499 6431 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:39.128656 6431 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:39.128976 6431 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:39.132112 6431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "40895a2710d84e019a88f5df904b1c79"
format_stamp: "Formatted at 2026-08-08 13:03:39 on dist-test-slave-t3q3"
server_key: "fa87300bbd6c45141541a33f796ece69"
server_key_iv: "ddb8e1707067caa0fc1bfdd3da6ad88c"
server_key_version: "encryptionkey@0"
I20260808 13:03:39.132982 6431 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "40895a2710d84e019a88f5df904b1c79"
format_stamp: "Formatted at 2026-08-08 13:03:39 on dist-test-slave-t3q3"
server_key: "fa87300bbd6c45141541a33f796ece69"
server_key_iv: "ddb8e1707067caa0fc1bfdd3da6ad88c"
server_key_version: "encryptionkey@0"
I20260808 13:03:39.138316 6431 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:03:39.141812 6465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:39.143168 6431 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:03:39.143354 6431 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "40895a2710d84e019a88f5df904b1c79"
format_stamp: "Formatted at 2026-08-08 13:03:39 on dist-test-slave-t3q3"
server_key: "fa87300bbd6c45141541a33f796ece69"
server_key_iv: "ddb8e1707067caa0fc1bfdd3da6ad88c"
server_key_version: "encryptionkey@0"
I20260808 13:03:39.143558 6431 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:39.158463 6431 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:39.163048 6431 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:39.163368 6431 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:39.164425 6431 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:03:39.165783 6431 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:39.165891 6431 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:39.166049 6431 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:39.166129 6431 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:39.202699 6431 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:45307
I20260808 13:03:39.202746 6643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:45307 every 8 connection(s)
I20260808 13:03:39.204121 6431 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:03:39.213418 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 6431
I20260808 13:03:39.213660 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:03:39.214150 1522 external_mini_cluster.cc:1428] Setting key d0ad1a2197466f3e3f6b89155344e443
Aug 08 13:03:39 dist-test-slave-t3q3 krb5kdc[6056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194219, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:39.226003 6647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:39.206881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:59565 (local address 127.1.124.190:39513)
0808 13:03:39.207352 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:39.207357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:39.208346 (+ 989us) server_negotiation.cc:407] Connection header received
0808 13:03:39.210209 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:39.210214 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:39.210313 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:39.210440 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:39.212123 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:39.212885 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:39.214800 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:39.215371 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:39.215503 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:39.219153 (+ 3650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:39.219158 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:39.219161 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:39.219194 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:39.222034 (+ 2840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:39.222845 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:39.222853 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:39.222859 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:39.223024 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:39.223550 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:39.223555 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:39.223558 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:39.223917 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:39.224165 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:39.225205 (+ 1040us) server_negotiation.cc:299] Negotiation successful
0808 13:03:39.225462 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":267,"threads_started":1}
I20260808 13:03:39.226819 6650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:39.207376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39513 (local address 127.1.124.130:59565)
0808 13:03:39.208176 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:39.208218 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:39.209905 (+ 1687us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:39.210700 (+ 795us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:39.210709 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:39.211283 (+ 574us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:39.211899 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:39.211919 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:39.213097 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:39.213102 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:39.214611 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:39.214626 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:39.215607 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:39.215612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:39.215818 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:39.216630 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:39.216658 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:39.218905 (+ 2247us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:39.222219 (+ 3314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:39.222230 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:39.222250 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:39.222710 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:39.223149 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:39.223154 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:39.223157 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:39.223415 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:39.224176 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:39.224186 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:39.224692 (+ 506us) client_negotiation.cc:769] Sending connection context
0808 13:03:39.225067 (+ 375us) client_negotiation.cc:240] Negotiation successful
0808 13:03:39.225640 (+ 573us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":282,"threads_started":1}
I20260808 13:03:39.228482 6644 heartbeater.cc:347] Connected to a master server at 127.1.124.190:39513
I20260808 13:03:39.228842 6644 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:39.229764 6644 heartbeater.cc:511] Master 127.1.124.190:39513 requested a full tablet report, sending...
I20260808 13:03:39.231483 6127 ts_manager.cc:194] Registered new tserver with Master: 40895a2710d84e019a88f5df904b1c79 (127.1.124.130:45307)
I20260808 13:03:39.232402 6127 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:59565
WARNING: no policy specified for kudu/127.1.124.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
I20260808 13:03:39.271371 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:39513
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:37875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:39.614675 6654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:39.614966 6654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:39.615075 6654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:39.622285 6654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:39.622462 6654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:39.622617 6654 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:03:39.632570 6654 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:37875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:39513
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:39.633853 6654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:39.635133 6654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:39.649107 6664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:39.649159 6666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:39.649189 6663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:39.653692 6654 server_base.cc:1034] running on GCE node
I20260808 13:03:39.654338 6654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:39.655311 6654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:39.656524 6654 hybrid_clock.cc:648] HybridClock initialized: now 1786194219656440 us; error 61 us; skew 500 ppm
I20260808 13:03:39.656869 6654 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:39 dist-test-slave-t3q3 krb5kdc[6056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194219, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:39.660115 6654 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:03:39.661553 6654 webserver.cc:460] Webserver started at http://127.1.124.131:46749/ using document root <none> and password file <none>
I20260808 13:03:39.662223 6654 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:39.662326 6654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:39.662585 6654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:39.665506 6654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "91e2ba6aeb1245cab1df70e2a864cf2d"
format_stamp: "Formatted at 2026-08-08 13:03:39 on dist-test-slave-t3q3"
server_key: "61e052787cfbb9eb0fc07c804c8ec573"
server_key_iv: "1d7dc5f5f1a5d9008e59ead71bf2db25"
server_key_version: "encryptionkey@0"
I20260808 13:03:39.666354 6654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "91e2ba6aeb1245cab1df70e2a864cf2d"
format_stamp: "Formatted at 2026-08-08 13:03:39 on dist-test-slave-t3q3"
server_key: "61e052787cfbb9eb0fc07c804c8ec573"
server_key_iv: "1d7dc5f5f1a5d9008e59ead71bf2db25"
server_key_version: "encryptionkey@0"
I20260808 13:03:39.671516 6654 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:03:39.675304 6674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:39.676819 6654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:03:39.677035 6654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "91e2ba6aeb1245cab1df70e2a864cf2d"
format_stamp: "Formatted at 2026-08-08 13:03:39 on dist-test-slave-t3q3"
server_key: "61e052787cfbb9eb0fc07c804c8ec573"
server_key_iv: "1d7dc5f5f1a5d9008e59ead71bf2db25"
server_key_version: "encryptionkey@0"
I20260808 13:03:39.677258 6654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:39.711679 6654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:39.716133 6654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:39.716399 6425 heartbeater.cc:503] Master 127.1.124.190:39513 was elected leader, sending a full tablet report...
I20260808 13:03:39.716490 6654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:39.717605 6654 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:03:39.719251 6654 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:39.719326 6654 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:39.719416 6654 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:39.719460 6654 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:39.753202 6654 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:45449
I20260808 13:03:39.753252 6842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:45449 every 8 connection(s)
I20260808 13:03:39.754524 6654 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:03:39.755789 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 6654
I20260808 13:03:39.755997 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:03:39.756567 1522 external_mini_cluster.cc:1428] Setting key 4bca785256d193c125ea56aa66a4ef59
Aug 08 13:03:39 dist-test-slave-t3q3 krb5kdc[6056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194219, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:39.776456 6845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:39.757430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:55241 (local address 127.1.124.190:39513)
0808 13:03:39.760384 (+ 2954us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:39.760388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:39.760404 (+ 16us) server_negotiation.cc:407] Connection header received
0808 13:03:39.761898 (+ 1494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:39.761902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:39.761974 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:39.762062 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:39.763493 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:39.764394 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:39.765972 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:39.766382 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:39.766475 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:39.769971 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:39.769976 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:39.769980 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:39.770017 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:39.773278 (+ 3261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:39.774012 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:39.774020 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:39.774025 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:39.774153 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:39.774593 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:39.774599 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:39.774602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:39.774879 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:39.775050 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:39.775780 (+ 730us) server_negotiation.cc:299] Negotiation successful
0808 13:03:39.776025 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2845,"thread_start_us":2685,"threads_started":1}
I20260808 13:03:39.777419 6847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:39.758015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39513 (local address 127.1.124.131:55241)
0808 13:03:39.759716 (+ 1701us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:39.759788 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:39.761656 (+ 1868us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:39.762273 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:39.762281 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:39.762800 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:39.763340 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:39.763357 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:39.764536 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:39.764540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:39.765818 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:39.765855 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:39.766568 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:39.766572 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:39.766778 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:39.767415 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:39.767442 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:39.769737 (+ 2295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:39.773425 (+ 3688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:39.773435 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:39.773451 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:39.773882 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:39.774278 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:39.774283 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:39.774287 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:39.774474 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:39.775018 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:39.775026 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:39.775394 (+ 368us) client_negotiation.cc:769] Sending connection context
0808 13:03:39.775692 (+ 298us) client_negotiation.cc:240] Negotiation successful
0808 13:03:39.776284 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1301,"thread_start_us":1047,"threads_started":1}
I20260808 13:03:39.778847 6843 heartbeater.cc:347] Connected to a master server at 127.1.124.190:39513
I20260808 13:03:39.779119 6843 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:39.779803 6843 heartbeater.cc:511] Master 127.1.124.190:39513 requested a full tablet report, sending...
I20260808 13:03:39.781231 6126 ts_manager.cc:194] Registered new tserver with Master: 91e2ba6aeb1245cab1df70e2a864cf2d (127.1.124.131:45449)
I20260808 13:03:39.782110 6126 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:55241
I20260808 13:03:39.785601 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 13:03:39 dist-test-slave-t3q3 krb5kdc[6056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194219, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 13:03:39 dist-test-slave-t3q3 krb5kdc[6056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194219, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:39.840140 6845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:39.825940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42258 (local address 127.1.124.190:39513)
0808 13:03:39.826160 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:39.826169 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:39.826559 (+ 390us) server_negotiation.cc:407] Connection header received
0808 13:03:39.826800 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:39.826805 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:39.826875 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:39.826985 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:39.828484 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:39.829257 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:39.830500 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:39.830911 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:39.831052 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:39.835071 (+ 4019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:39.835076 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:39.835079 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:39.835129 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:39.837656 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:39.838289 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:39.838293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:39.838295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:39.838413 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:39.838775 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:39.838778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:39.838780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:39.839090 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:39.839283 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:39.839523 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 13:03:39.839702 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 13:03:39.840709 6126 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42258
I20260808 13:03:39.842275 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 6226
Aug 08 13:03:39 dist-test-slave-t3q3 krb5kdc[6056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194219, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.124.129@KRBTEST.COM
I20260808 13:03:39.858503 6859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:39.843545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41264 (local address 127.1.124.129:38047)
0808 13:03:39.844249 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:39.844256 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:39.844281 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:03:39.844348 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:39.844354 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:39.844579 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:39.844709 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:39.845896 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:39.847023 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:39.848412 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:39.848976 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:39.849091 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:39.853055 (+ 3964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:39.853067 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:39.853081 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:39.853115 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:39.855952 (+ 2837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:39.856679 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:39.856686 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:39.856688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:39.856831 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:39.857175 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:39.857179 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:39.857181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:39.857523 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:39.857768 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:39.858006 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 13:03:39.858212 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":355,"threads_started":1}
W20260808 13:03:39.859885 6414 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:41264
W20260808 13:03:39.861130 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 6226: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:03:39.861336 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 6226
I20260808 13:03:39.885481 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 6431
Aug 08 13:03:39 dist-test-slave-t3q3 krb5kdc[6056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194219, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.124.130@KRBTEST.COM
I20260808 13:03:39.901043 6863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:39.886333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35528 (local address 127.1.124.130:45307)
0808 13:03:39.886897 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:39.886913 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:39.886940 (+ 27us) server_negotiation.cc:407] Connection header received
0808 13:03:39.887003 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:39.887009 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:39.887240 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:39.887376 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:39.888671 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:39.889722 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:39.890889 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:39.891436 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:39.891529 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:39.895780 (+ 4251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:39.895791 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:39.895806 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:39.895838 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:39.898573 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:39.899171 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:39.899178 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:39.899180 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:39.899302 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:39.899613 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:39.899617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:39.899618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:39.899926 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:39.900222 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:39.900556 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 13:03:39.900752 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":292,"threads_started":1}
W20260808 13:03:39.902407 6636 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:35528
W20260808 13:03:39.903544 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 6431: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:03:39.903718 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 6431
I20260808 13:03:39.925347 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 6654
Aug 08 13:03:39 dist-test-slave-t3q3 krb5kdc[6056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194219, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.1.124.131@KRBTEST.COM
I20260808 13:03:39.943307 6864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:39.926492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54460 (local address 127.1.124.131:45449)
0808 13:03:39.927149 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:39.927157 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:39.927187 (+ 30us) server_negotiation.cc:407] Connection header received
0808 13:03:39.927253 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:39.927260 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:39.927468 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:39.927626 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:39.929252 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:39.930375 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:39.931798 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:39.932578 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:39.932700 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:39.937279 (+ 4579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:39.937293 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:39.937308 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:39.937344 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:39.940372 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:39.941077 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:39.941084 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:39.941087 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:39.941224 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:39.941618 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:39.941626 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:39.941629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:39.941998 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:39.942344 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:39.942631 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 13:03:39.942925 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":503,"thread_start_us":328,"threads_started":1}
W20260808 13:03:39.944734 6837 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54460
W20260808 13:03:39.945990 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 6654: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:03:39.946162 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 6654
I20260808 13:03:39.971424 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 6077
I20260808 13:03:39.972723 6175 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:40.080096 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 6077
2026-08-08T13:03:40Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (4076 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:03:40 dist-test-slave-t3q3 krb5kdc[6869](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:03:40 dist-test-slave-t3q3 krb5kdc[6869](info): set up 2 sockets
Aug 08 13:03:40 dist-test-slave-t3q3 krb5kdc[6869](info): commencing operation
krb5kdc: starting...
W20260808 13:03:41.414781 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.282s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:03:41 dist-test-slave-t3q3 krb5kdc[6869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:03:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:03:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:03:41.655790 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:34617
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:40329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:34617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:41.989320 6885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:41.989688 6885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:41.989846 6885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:41.997206 6885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:03:41.997367 6885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:41.997478 6885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:41.997581 6885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:03:41.997673 6885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:03:42.007843 6885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:40329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:34617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:34617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:42.009321 6885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:42.010689 6885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:42.023025 6896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:42.023110 6902 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:42.023057 6899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:42.024179 6885 server_base.cc:1034] running on GCE node
I20260808 13:03:42.024999 6885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:42.026448 6885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:42.027683 6885 hybrid_clock.cc:648] HybridClock initialized: now 1786194222027632 us; error 42 us; skew 500 ppm
I20260808 13:03:42.028091 6885 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:42 dist-test-slave-t3q3 krb5kdc[6869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194222, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:42.031330 6885 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:03:42.033003 6885 webserver.cc:460] Webserver started at http://127.1.124.190:37107/ using document root <none> and password file <none>
I20260808 13:03:42.033715 6885 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:42.033861 6885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:42.034196 6885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:42.037297 6885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "ec74ea31eba24a5788962440be975260"
format_stamp: "Formatted at 2026-08-08 13:03:42 on dist-test-slave-t3q3"
server_key: "e6c58c64699d5f8c033959f7f29d4bd6"
server_key_iv: "a931634242d081b081f3b89d37fd7979"
server_key_version: "encryptionkey@0"
I20260808 13:03:42.038134 6885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "ec74ea31eba24a5788962440be975260"
format_stamp: "Formatted at 2026-08-08 13:03:42 on dist-test-slave-t3q3"
server_key: "e6c58c64699d5f8c033959f7f29d4bd6"
server_key_iv: "a931634242d081b081f3b89d37fd7979"
server_key_version: "encryptionkey@0"
I20260808 13:03:42.043491 6885 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 13:03:42.047631 6916 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:42.049008 6885 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:03:42.049211 6885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "ec74ea31eba24a5788962440be975260"
format_stamp: "Formatted at 2026-08-08 13:03:42 on dist-test-slave-t3q3"
server_key: "e6c58c64699d5f8c033959f7f29d4bd6"
server_key_iv: "a931634242d081b081f3b89d37fd7979"
server_key_version: "encryptionkey@0"
I20260808 13:03:42.049410 6885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:42.063799 6885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:42.067798 6885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:42.068148 6885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:42.087404 6885 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:34617
I20260808 13:03:42.087440 6988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:34617 every 8 connection(s)
I20260808 13:03:42.088917 6885 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:03:42.092684 6990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:42.098856 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 6885
I20260808 13:03:42.099077 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:03:42.099704 1522 external_mini_cluster.cc:1428] Setting key ccefa64e43b775a6291373ddd8b761fc
I20260808 13:03:42.103574 6990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260: Bootstrap starting.
I20260808 13:03:42.107425 6990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:42.108599 6990 log.cc:824] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:42.111747 6990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260: No bootstrap required, opened a new log
Aug 08 13:03:42 dist-test-slave-t3q3 krb5kdc[6869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194221, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:42.120556 6990 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec74ea31eba24a5788962440be975260" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 34617 } }
I20260808 13:03:42.120965 6990 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:42.121093 6990 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec74ea31eba24a5788962440be975260, State: Initialized, Role: FOLLOWER
I20260808 13:03:42.121742 6990 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec74ea31eba24a5788962440be975260" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 34617 } }
I20260808 13:03:42.122078 6990 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:03:42.122233 6990 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:03:42.122423 6990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:42.125346 6996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:42.101862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59232 (local address 127.1.124.190:34617)
0808 13:03:42.102662 (+ 800us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:42.102674 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:42.102715 (+ 41us) server_negotiation.cc:407] Connection header received
0808 13:03:42.104525 (+ 1810us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:42.104548 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:42.104960 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:42.105309 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:42.107279 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:42.108862 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:42.110544 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:42.111210 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:42.111345 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:42.116466 (+ 5121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:42.116475 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:42.116492 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:42.116538 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:42.120286 (+ 3748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:42.121099 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:42.121111 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:42.121121 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:42.121352 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:42.121829 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:42.121834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:42.121838 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:42.122664 (+ 826us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:42.122957 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:42.123408 (+ 451us) server_negotiation.cc:299] Negotiation successful
0808 13:03:42.123880 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":365,"threads_started":1}
I20260808 13:03:42.125944 6990 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec74ea31eba24a5788962440be975260" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 34617 } }
I20260808 13:03:42.126566 6990 leader_election.cc:302] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec74ea31eba24a5788962440be975260; no voters:
I20260808 13:03:42.127271 6990 leader_election.cc:288] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:03:42.127471 7000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:42.129658 6990 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:03:42.130019 7000 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260 [term 1 LEADER]: Becoming Leader. State: Replica: ec74ea31eba24a5788962440be975260, State: Running, Role: LEADER
I20260808 13:03:42.130549 7000 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec74ea31eba24a5788962440be975260" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 34617 } }
I20260808 13:03:42.133688 7003 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec74ea31eba24a5788962440be975260. Latest consensus state: current_term: 1 leader_uuid: "ec74ea31eba24a5788962440be975260" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec74ea31eba24a5788962440be975260" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 34617 } } }
I20260808 13:03:42.134155 7003 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260 [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:42.134071 7001 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec74ea31eba24a5788962440be975260" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec74ea31eba24a5788962440be975260" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 34617 } } }
I20260808 13:03:42.134339 7001 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260 [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:42.135474 7018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:03:42.140795 7018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:03:42.149972 7018 catalog_manager.cc:1347] Generated new cluster ID: 2c18fb99c1f94eb998b342d35d463442
I20260808 13:03:42.150118 7018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:03:42.164197 7018 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:03:42.165485 7018 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:03:42.173982 7018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec74ea31eba24a5788962440be975260: Generated new TSK 0
I20260808 13:03:42.174821 7018 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.124.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
I20260808 13:03:42.244006 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:34617
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:40329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:42.591979 7036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:42.592358 7036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:42.592564 7036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:42.600589 7036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:42.600694 7036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:42.600806 7036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:03:42.610545 7036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:40329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:34617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:42.611778 7036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:42.613212 7036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:42.626330 7044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:42.626305 7045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:42.626271 7051 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:42.627353 7036 server_base.cc:1034] running on GCE node
I20260808 13:03:42.627949 7036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:42.628947 7036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:42.630167 7036 hybrid_clock.cc:648] HybridClock initialized: now 1786194222630117 us; error 38 us; skew 500 ppm
I20260808 13:03:42.630520 7036 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:42 dist-test-slave-t3q3 krb5kdc[6869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194222, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:42.634011 7036 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:03:42.635814 7036 webserver.cc:460] Webserver started at http://127.1.124.129:40991/ using document root <none> and password file <none>
I20260808 13:03:42.636613 7036 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:42.636759 7036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:42.637131 7036 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:42.640264 7036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "74d158f1cc2e4101839174b40e45a3af"
format_stamp: "Formatted at 2026-08-08 13:03:42 on dist-test-slave-t3q3"
server_key: "3ac83e1aa246308252a62da8ce7a2d92"
server_key_iv: "fbf99fc6e62bb3ca51d1d1c850a33122"
server_key_version: "encryptionkey@0"
I20260808 13:03:42.641098 7036 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "74d158f1cc2e4101839174b40e45a3af"
format_stamp: "Formatted at 2026-08-08 13:03:42 on dist-test-slave-t3q3"
server_key: "3ac83e1aa246308252a62da8ce7a2d92"
server_key_iv: "fbf99fc6e62bb3ca51d1d1c850a33122"
server_key_version: "encryptionkey@0"
I20260808 13:03:42.646551 7036 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 13:03:42.650329 7060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:42.651718 7036 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:03:42.651912 7036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "74d158f1cc2e4101839174b40e45a3af"
format_stamp: "Formatted at 2026-08-08 13:03:42 on dist-test-slave-t3q3"
server_key: "3ac83e1aa246308252a62da8ce7a2d92"
server_key_iv: "fbf99fc6e62bb3ca51d1d1c850a33122"
server_key_version: "encryptionkey@0"
I20260808 13:03:42.652185 7036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:42.668380 7036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:42.672675 7036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:42.672991 7036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:42.673965 7036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:03:42.675387 7036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:42.675462 7036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:42.675634 7036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:42.675695 7036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:42.710345 7036 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:43891
I20260808 13:03:42.710394 7227 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:43891 every 8 connection(s)
I20260808 13:03:42.711787 7036 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:03:42.721653 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 7036
I20260808 13:03:42.721884 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:03:42.722551 1522 external_mini_cluster.cc:1428] Setting key 10e21430886c1aa8788c0782e45007b8
Aug 08 13:03:42 dist-test-slave-t3q3 krb5kdc[6869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194222, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:42.736377 7232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:42.714778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:54629 (local address 127.1.124.190:34617)
0808 13:03:42.715279 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:42.715284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:42.716434 (+ 1150us) server_negotiation.cc:407] Connection header received
0808 13:03:42.718292 (+ 1858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:42.718298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:42.718394 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:42.718520 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:42.720119 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:42.720865 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:42.722630 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:42.723107 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:42.723279 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:42.727456 (+ 4177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:42.727465 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:42.727473 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:42.727520 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:42.731608 (+ 4088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:42.732784 (+ 1176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:42.732793 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:42.732797 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:42.733085 (+ 288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:42.733745 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:42.733753 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:42.733757 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:42.734213 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:42.734419 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:42.735600 (+ 1181us) server_negotiation.cc:299] Negotiation successful
0808 13:03:42.735903 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":250,"threads_started":1}
I20260808 13:03:42.737694 7235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:42.715397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:34617 (local address 127.1.124.129:54629)
0808 13:03:42.716258 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:42.716310 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:42.718008 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:42.718787 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:42.718795 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:42.719322 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:42.719914 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:42.719932 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:42.721046 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:42.721051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:42.722474 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:42.722487 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:42.723377 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:42.723386 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:42.723694 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:42.724514 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:42.724537 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:42.727220 (+ 2683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:42.731810 (+ 4590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:42.731823 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:42.731844 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:42.732571 (+ 727us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:42.733264 (+ 693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:42.733271 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:42.733278 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:42.733581 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:42.734413 (+ 832us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:42.734422 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:42.735095 (+ 673us) client_negotiation.cc:769] Sending connection context
0808 13:03:42.735439 (+ 344us) client_negotiation.cc:240] Negotiation successful
0808 13:03:42.736052 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":306,"threads_started":1}
I20260808 13:03:42.739972 7229 heartbeater.cc:347] Connected to a master server at 127.1.124.190:34617
I20260808 13:03:42.740357 7229 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:42.741165 7229 heartbeater.cc:511] Master 127.1.124.190:34617 requested a full tablet report, sending...
I20260808 13:03:42.743359 6942 ts_manager.cc:194] Registered new tserver with Master: 74d158f1cc2e4101839174b40e45a3af (127.1.124.129:43891)
I20260808 13:03:42.745044 6942 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:54629
WARNING: no policy specified for kudu/127.1.124.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
I20260808 13:03:42.781176 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:34617
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:40329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:43.117022 7240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:43.117345 7240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:43.117515 7240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:43.124850 7240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:43.125017 7240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:43.125200 7240 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:03:43.135577 7240 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:40329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:34617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:43.136951 7240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:43.138302 7240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:43.151587 7248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:43.151677 7249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:43.151618 7253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:43.152428 7240 server_base.cc:1034] running on GCE node
I20260808 13:03:43.153214 7240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:43.154145 7240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:43.155383 7240 hybrid_clock.cc:648] HybridClock initialized: now 1786194223155323 us; error 48 us; skew 500 ppm
I20260808 13:03:43.155766 7240 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:43 dist-test-slave-t3q3 krb5kdc[6869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194223, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:43.163712 7240 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:03:43.165246 7240 webserver.cc:460] Webserver started at http://127.1.124.130:39379/ using document root <none> and password file <none>
I20260808 13:03:43.165903 7240 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:43.166008 7240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:43.166280 7240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:43.169246 7240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "f4fa2b9a1ebf4e62970ff9fbc5cd1c78"
format_stamp: "Formatted at 2026-08-08 13:03:43 on dist-test-slave-t3q3"
server_key: "04d8acd18a52f294b7710f7c51fde7ef"
server_key_iv: "c525f6135d764c5e7f00e79289e82f91"
server_key_version: "encryptionkey@0"
I20260808 13:03:43.170014 7240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "f4fa2b9a1ebf4e62970ff9fbc5cd1c78"
format_stamp: "Formatted at 2026-08-08 13:03:43 on dist-test-slave-t3q3"
server_key: "04d8acd18a52f294b7710f7c51fde7ef"
server_key_iv: "c525f6135d764c5e7f00e79289e82f91"
server_key_version: "encryptionkey@0"
I20260808 13:03:43.175266 7240 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 13:03:43.178701 7267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:43.180140 7240 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:03:43.180321 7240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "f4fa2b9a1ebf4e62970ff9fbc5cd1c78"
format_stamp: "Formatted at 2026-08-08 13:03:43 on dist-test-slave-t3q3"
server_key: "04d8acd18a52f294b7710f7c51fde7ef"
server_key_iv: "c525f6135d764c5e7f00e79289e82f91"
server_key_version: "encryptionkey@0"
I20260808 13:03:43.180531 7240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:43.202860 7240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:43.207378 7240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:43.207723 7240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:43.208837 7240 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:03:43.210421 7240 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:43.210542 7240 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:43.210705 7240 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:43.210783 7240 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:43.246680 7240 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:35791
I20260808 13:03:43.246721 7439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:35791 every 8 connection(s)
I20260808 13:03:43.247982 7240 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:03:43.254877 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 7240
I20260808 13:03:43.255060 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:03:43.255620 1522 external_mini_cluster.cc:1428] Setting key 2ef286fba078d8be9d5b25567bd7cdc5
Aug 08 13:03:43 dist-test-slave-t3q3 krb5kdc[6869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194223, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:43.270370 7442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:43.250603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:33401 (local address 127.1.124.190:34617)
0808 13:03:43.251105 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:43.251113 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:43.252094 (+ 981us) server_negotiation.cc:407] Connection header received
0808 13:03:43.254063 (+ 1969us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:43.254067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:43.254173 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:43.254274 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:43.256345 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:43.257252 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:43.258947 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:43.259785 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:43.259959 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:43.263782 (+ 3823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:43.263790 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:43.263797 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:43.263844 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:43.267032 (+ 3188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:43.267982 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:43.267986 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:43.267988 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:43.268198 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:43.268689 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:43.268692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:43.268694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:43.268942 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:43.269050 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:43.269932 (+ 882us) server_negotiation.cc:299] Negotiation successful
0808 13:03:43.270094 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":233,"threads_started":1}
I20260808 13:03:43.272082 7443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:43.251171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:34617 (local address 127.1.124.130:33401)
0808 13:03:43.251896 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:43.251942 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:43.253792 (+ 1850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:43.254525 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:43.254538 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:43.255315 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:43.256138 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:43.256165 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:43.257422 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:43.257426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:43.258790 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:43.258802 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:43.260122 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:43.260131 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:43.260357 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:43.261086 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:43.261114 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:43.263531 (+ 2417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:43.267184 (+ 3653us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:43.267196 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:43.267215 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:43.267860 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:43.268307 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:43.268315 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:43.268322 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:43.268575 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:43.269076 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:43.269086 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:43.269543 (+ 457us) client_negotiation.cc:769] Sending connection context
0808 13:03:43.269872 (+ 329us) client_negotiation.cc:240] Negotiation successful
0808 13:03:43.270488 (+ 616us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"thread_start_us":268,"threads_started":1}
I20260808 13:03:43.273775 7440 heartbeater.cc:347] Connected to a master server at 127.1.124.190:34617
I20260808 13:03:43.274114 7440 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:43.275019 7440 heartbeater.cc:511] Master 127.1.124.190:34617 requested a full tablet report, sending...
I20260808 13:03:43.276847 6942 ts_manager.cc:194] Registered new tserver with Master: f4fa2b9a1ebf4e62970ff9fbc5cd1c78 (127.1.124.130:35791)
I20260808 13:03:43.278151 6942 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:33401
WARNING: no policy specified for kudu/127.1.124.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
I20260808 13:03:43.311690 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:34617
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:40329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:43.643749 7447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:43.644037 7447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:43.644258 7447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:43.651417 7447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:43.651523 7447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:43.651738 7447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:03:43.661541 7447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:40329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:34617
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:43.662827 7447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:43.664207 7447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:43.677166 7462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:43.677244 7459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:43.677279 7456 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:43.678469 7447 server_base.cc:1034] running on GCE node
I20260808 13:03:43.679100 7447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:43.680053 7447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:43.681329 7447 hybrid_clock.cc:648] HybridClock initialized: now 1786194223681280 us; error 55 us; skew 500 ppm
I20260808 13:03:43.681663 7447 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:43 dist-test-slave-t3q3 krb5kdc[6869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194223, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:43.685297 7447 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:03:43.687036 7447 webserver.cc:460] Webserver started at http://127.1.124.131:37395/ using document root <none> and password file <none>
I20260808 13:03:43.687914 7447 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:43.688099 7447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:43.688489 7447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:43.691599 7447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "c37797445d604bf7aea61061fbfd27dd"
format_stamp: "Formatted at 2026-08-08 13:03:43 on dist-test-slave-t3q3"
server_key: "48b5094dc204ec613da3075d11d19455"
server_key_iv: "fc561e0477205f1a9f69350c924f4761"
server_key_version: "encryptionkey@0"
I20260808 13:03:43.692526 7447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "c37797445d604bf7aea61061fbfd27dd"
format_stamp: "Formatted at 2026-08-08 13:03:43 on dist-test-slave-t3q3"
server_key: "48b5094dc204ec613da3075d11d19455"
server_key_iv: "fc561e0477205f1a9f69350c924f4761"
server_key_version: "encryptionkey@0"
I20260808 13:03:43.698045 7447 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 13:03:43.701581 7473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:43.702913 7447 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:03:43.703104 7447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "c37797445d604bf7aea61061fbfd27dd"
format_stamp: "Formatted at 2026-08-08 13:03:43 on dist-test-slave-t3q3"
server_key: "48b5094dc204ec613da3075d11d19455"
server_key_iv: "fc561e0477205f1a9f69350c924f4761"
server_key_version: "encryptionkey@0"
I20260808 13:03:43.703326 7447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:43.735706 7447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:43.739984 7447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:43.740316 7447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:43.741338 7447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:03:43.742754 7447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:43.742861 7447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:43.743018 7447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:43.743093 7447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:43.748224 7229 heartbeater.cc:503] Master 127.1.124.190:34617 was elected leader, sending a full tablet report...
I20260808 13:03:43.779501 7447 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:42803
I20260808 13:03:43.779593 7652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:42803 every 8 connection(s)
I20260808 13:03:43.781391 7447 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:03:43.784706 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 7447
I20260808 13:03:43.784932 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:03:43.785634 1522 external_mini_cluster.cc:1428] Setting key 629f2367e82ec64b17892d773bfbbe7f
Aug 08 13:03:43 dist-test-slave-t3q3 krb5kdc[6869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194223, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:43.805627 7656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:43.786653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:43303 (local address 127.1.124.190:34617)
0808 13:03:43.787308 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:43.787319 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:43.790580 (+ 3261us) server_negotiation.cc:407] Connection header received
0808 13:03:43.790649 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:43.790652 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:43.790778 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:43.790887 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:43.792868 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:43.793866 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:43.795456 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:43.795875 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:43.796013 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:43.799556 (+ 3543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:43.799561 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:43.799565 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:43.799596 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:43.802468 (+ 2872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:43.803228 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:43.803233 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:43.803235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:43.803355 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:43.803800 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:43.803805 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:43.803810 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:43.804137 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:43.804302 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:43.805004 (+ 702us) server_negotiation.cc:299] Negotiation successful
0808 13:03:43.805226 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":255,"threads_started":1}
I20260808 13:03:43.806692 7657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:43.787383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:34617 (local address 127.1.124.131:43303)
0808 13:03:43.788350 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:43.788397 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:43.790252 (+ 1855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:43.791155 (+ 903us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:43.791187 (+ 32us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:43.791788 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:43.792633 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:43.792665 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:43.794021 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:43.794024 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:43.795313 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:43.795325 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:43.796145 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:43.796153 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:43.796367 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:43.797098 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:43.797120 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:43.799355 (+ 2235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:43.802625 (+ 3270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:43.802634 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:43.802651 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:43.803090 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:43.803467 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:43.803472 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:43.803476 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:43.803676 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:43.804264 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:43.804272 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:43.804626 (+ 354us) client_negotiation.cc:769] Sending connection context
0808 13:03:43.804902 (+ 276us) client_negotiation.cc:240] Negotiation successful
0808 13:03:43.805448 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"thread_start_us":414,"threads_started":1}
I20260808 13:03:43.808287 7653 heartbeater.cc:347] Connected to a master server at 127.1.124.190:34617
I20260808 13:03:43.808629 7653 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:43.809376 7653 heartbeater.cc:511] Master 127.1.124.190:34617 requested a full tablet report, sending...
I20260808 13:03:43.810810 6942 ts_manager.cc:194] Registered new tserver with Master: c37797445d604bf7aea61061fbfd27dd (127.1.124.131:42803)
I20260808 13:03:43.811617 6942 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:43303
I20260808 13:03:43.814836 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 13:03:43 dist-test-slave-t3q3 krb5kdc[6869](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 13:03:43 dist-test-slave-t3q3 krb5kdc[6869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.129@KRBTEST.COM
I20260808 13:03:43.872176 7668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:43.856458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38690 (local address 127.1.124.129:43891)
0808 13:03:43.857221 (+ 763us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:43.857228 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:43.857256 (+ 28us) server_negotiation.cc:407] Connection header received
0808 13:03:43.857327 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:43.857334 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:43.857547 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:43.857686 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:43.859157 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:43.860267 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:43.861472 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:43.862043 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:43.862142 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:43.866137 (+ 3995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:43.866147 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:43.866162 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:43.866198 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:43.869163 (+ 2965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:43.869851 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:43.869858 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:43.869860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:43.869984 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:43.870398 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:43.870404 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:43.870407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:43.870853 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:43.871169 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:43.871469 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 13:03:43.871751 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":345,"threads_started":1}
Aug 08 13:03:43 dist-test-slave-t3q3 krb5kdc[6869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:43.891029 7656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:43.876685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59244 (local address 127.1.124.190:34617)
0808 13:03:43.876923 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:43.876928 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:43.877030 (+ 102us) server_negotiation.cc:407] Connection header received
0808 13:03:43.877172 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:43.877176 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:43.877254 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:43.877336 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:43.878515 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:43.879287 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:43.880589 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:43.881010 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:43.881119 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:43.885356 (+ 4237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:43.885361 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:43.885364 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:43.885398 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:43.888273 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:43.888968 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:43.888974 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:43.888978 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:43.889114 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:43.889506 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:43.889511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:43.889515 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:43.889866 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:43.890126 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:43.890477 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 13:03:43.890705 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
W20260808 13:03:43.891691 6942 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59244
I20260808 13:03:43.892884 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 7036
I20260808 13:03:43.906643 7668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:43.894049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38698 (local address 127.1.124.129:43891)
0808 13:03:43.894274 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:43.894279 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:43.894411 (+ 132us) server_negotiation.cc:407] Connection header received
0808 13:03:43.894617 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:43.894627 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:43.894694 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:43.894802 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:43.896016 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:43.896765 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:43.898041 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:43.898584 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:43.898735 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:43.901203 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:43.901208 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:43.901211 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:43.901245 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:43.903856 (+ 2611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:43.904626 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:43.904632 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:43.904636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:43.904776 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:43.905170 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:43.905176 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:43.905180 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:43.905449 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:43.905718 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:43.906039 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 13:03:43.906268 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 13:03:43.907308 7222 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:43.991539 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 7036
I20260808 13:03:44.017540 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 7240
Aug 08 13:03:44 dist-test-slave-t3q3 krb5kdc[6869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.130@KRBTEST.COM
I20260808 13:03:44.034474 7680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:44.018480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48394 (local address 127.1.124.130:35791)
0808 13:03:44.019039 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:44.019047 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:44.019074 (+ 27us) server_negotiation.cc:407] Connection header received
0808 13:03:44.019139 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:44.019146 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:44.019361 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:44.019495 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:44.020876 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:44.022196 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:44.023498 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:44.024245 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:44.024355 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:44.028741 (+ 4386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:44.028752 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:44.028767 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:44.028818 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:44.031661 (+ 2843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:44.032402 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:44.032409 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:44.032412 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:44.032547 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:44.033032 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:44.033035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:44.033037 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:44.033374 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:44.033617 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:44.033967 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 13:03:44.034183 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":294,"threads_started":1}
I20260808 13:03:44.035799 7435 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:44.119549 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 7240
I20260808 13:03:44.145509 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 7447
Aug 08 13:03:44 dist-test-slave-t3q3 krb5kdc[6869](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194223, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.131@KRBTEST.COM
I20260808 13:03:44.161783 7683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:44.146425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38998 (local address 127.1.124.131:42803)
0808 13:03:44.147055 (+ 630us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:44.147063 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:44.147091 (+ 28us) server_negotiation.cc:407] Connection header received
0808 13:03:44.147156 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:44.147162 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:44.147371 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:44.147548 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:44.148951 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:44.149993 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:44.151284 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:44.151862 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:44.152021 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:44.156573 (+ 4552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:44.156584 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:44.156599 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:44.156631 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:44.159287 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:44.159885 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:44.159891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:44.159893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:44.160008 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:44.160378 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:44.160381 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:44.160383 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:44.160724 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:44.160995 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:44.161177 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 13:03:44.161384 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":312,"threads_started":1}
I20260808 13:03:44.163057 7644 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:44.247167 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 7447
I20260808 13:03:44.272360 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 6885
I20260808 13:03:44.273694 6984 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:44.378041 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 6885
2026-08-08T13:03:44Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4297 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:03:44 dist-test-slave-t3q3 krb5kdc[7689](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:03:44 dist-test-slave-t3q3 krb5kdc[7689](info): set up 2 sockets
Aug 08 13:03:44 dist-test-slave-t3q3 krb5kdc[7689](info): commencing operation
krb5kdc: starting...
W20260808 13:03:45.701284 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.004s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:03:45 dist-test-slave-t3q3 krb5kdc[7689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:03:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:03:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:03:45.940356 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:34977
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:44949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:34977
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:46.279711 7706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:46.280118 7706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:46.280297 7706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:46.287904 7706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:03:46.288116 7706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:46.288236 7706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:46.288344 7706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:03:46.288429 7706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:03:46.298828 7706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:44949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:34977
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:34977
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:46.300367 7706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:46.301903 7706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:46.314320 7717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:46.314311 7714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:46.314335 7715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:46.315083 7706 server_base.cc:1034] running on GCE node
I20260808 13:03:46.315946 7706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:46.317423 7706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:46.318673 7706 hybrid_clock.cc:648] HybridClock initialized: now 1786194226318632 us; error 35 us; skew 500 ppm
I20260808 13:03:46.319044 7706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:46 dist-test-slave-t3q3 krb5kdc[7689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194226, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:46.322319 7706 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:03:46.323518 7706 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:46.323675 7706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:46.324014 7706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:46.327104 7706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "63f435c0b1b94dfb9a2ecbdeef3c11cf"
format_stamp: "Formatted at 2026-08-08 13:03:46 on dist-test-slave-t3q3"
server_key: "06646f05f8bcffaa6b3eb2889877bcd0"
server_key_iv: "e0fc36e6acd52b42ab8109466743cfa0"
server_key_version: "encryptionkey@0"
I20260808 13:03:46.327965 7706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "63f435c0b1b94dfb9a2ecbdeef3c11cf"
format_stamp: "Formatted at 2026-08-08 13:03:46 on dist-test-slave-t3q3"
server_key: "06646f05f8bcffaa6b3eb2889877bcd0"
server_key_iv: "e0fc36e6acd52b42ab8109466743cfa0"
server_key_version: "encryptionkey@0"
I20260808 13:03:46.333461 7706 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:03:46.337177 7727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:46.338567 7706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 13:03:46.338762 7706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "63f435c0b1b94dfb9a2ecbdeef3c11cf"
format_stamp: "Formatted at 2026-08-08 13:03:46 on dist-test-slave-t3q3"
server_key: "06646f05f8bcffaa6b3eb2889877bcd0"
server_key_iv: "e0fc36e6acd52b42ab8109466743cfa0"
server_key_version: "encryptionkey@0"
I20260808 13:03:46.339000 7706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:46.361641 7706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:46.366297 7706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:46.366588 7706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:46.385361 7706 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:34977
I20260808 13:03:46.385382 7814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:34977 every 8 connection(s)
I20260808 13:03:46.386334 7706 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:03:46.390161 7815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:46.394780 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 7706
I20260808 13:03:46.395027 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:03:46.395676 1522 external_mini_cluster.cc:1428] Setting key 2c4e452fd296d580411498a2b25d96fa
I20260808 13:03:46.402845 7815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf: Bootstrap starting.
I20260808 13:03:46.406571 7815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:46.407796 7815 log.cc:824] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf: Log is configured to *not* fsync() on all Append() calls
Aug 08 13:03:46 dist-test-slave-t3q3 krb5kdc[7689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:46.411300 7815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf: No bootstrap required, opened a new log
I20260808 13:03:46.419100 7815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63f435c0b1b94dfb9a2ecbdeef3c11cf" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 34977 } }
I20260808 13:03:46.419534 7815 raft_consensus.cc:374] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:46.419665 7815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63f435c0b1b94dfb9a2ecbdeef3c11cf, State: Initialized, Role: FOLLOWER
I20260808 13:03:46.419898 7820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:46.397808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34734 (local address 127.1.124.190:34977)
0808 13:03:46.398626 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:46.398637 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:46.398681 (+ 44us) server_negotiation.cc:407] Connection header received
0808 13:03:46.400795 (+ 2114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:46.400819 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:46.401203 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:46.401513 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:46.403662 (+ 2149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:46.405083 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:46.406910 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:46.407585 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:46.407718 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:46.412355 (+ 4637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:46.412363 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:46.412377 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:46.412416 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:46.415546 (+ 3130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:46.416341 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:46.416348 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:46.416354 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:46.416531 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:46.416923 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:46.416928 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:46.416933 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:46.417506 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:46.417841 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:46.418298 (+ 457us) server_negotiation.cc:299] Negotiation successful
0808 13:03:46.418724 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":418,"threads_started":1}
I20260808 13:03:46.420342 7815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63f435c0b1b94dfb9a2ecbdeef3c11cf" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 34977 } }
I20260808 13:03:46.420624 7815 raft_consensus.cc:388] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:03:46.420783 7815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:03:46.420982 7815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:46.423534 7815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63f435c0b1b94dfb9a2ecbdeef3c11cf" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 34977 } }
I20260808 13:03:46.424130 7815 leader_election.cc:302] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 63f435c0b1b94dfb9a2ecbdeef3c11cf; no voters:
I20260808 13:03:46.424726 7815 leader_election.cc:288] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:03:46.424857 7822 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:46.425629 7822 raft_consensus.cc:686] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf [term 1 LEADER]: Becoming Leader. State: Replica: 63f435c0b1b94dfb9a2ecbdeef3c11cf, State: Running, Role: LEADER
I20260808 13:03:46.426189 7822 consensus_queue.cc:237] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63f435c0b1b94dfb9a2ecbdeef3c11cf" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 34977 } }
I20260808 13:03:46.427184 7815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:03:46.429256 7824 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 63f435c0b1b94dfb9a2ecbdeef3c11cf. Latest consensus state: current_term: 1 leader_uuid: "63f435c0b1b94dfb9a2ecbdeef3c11cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63f435c0b1b94dfb9a2ecbdeef3c11cf" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 34977 } } }
I20260808 13:03:46.429658 7824 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:46.429251 7823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "63f435c0b1b94dfb9a2ecbdeef3c11cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "63f435c0b1b94dfb9a2ecbdeef3c11cf" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 34977 } } }
I20260808 13:03:46.429776 7823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:46.432605 7832 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:03:46.437974 7832 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:03:46.448050 7832 catalog_manager.cc:1347] Generated new cluster ID: 8020036d4a63489f9bc01cdcfba90d51
I20260808 13:03:46.448237 7832 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:03:46.470954 7832 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:03:46.472440 7832 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:03:46.478412 7832 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf: Generated new TSK 0
I20260808 13:03:46.479265 7832 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.124.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
I20260808 13:03:46.547499 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:34977
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:44949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:46.891392 7854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:46.891698 7854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:46.891803 7854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:46.899489 7854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:46.899609 7854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:46.899735 7854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:03:46.910326 7854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:44949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:34977
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:46.911645 7854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:46.913058 7854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:46.926455 7860 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:46.926565 7861 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:46.926986 7864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:46.927744 7854 server_base.cc:1034] running on GCE node
I20260808 13:03:46.928490 7854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:46.929466 7854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:46.930716 7854 hybrid_clock.cc:648] HybridClock initialized: now 1786194226930667 us; error 44 us; skew 500 ppm
I20260808 13:03:46.931006 7854 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:46 dist-test-slave-t3q3 krb5kdc[7689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194226, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:46.934453 7854 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:03:46.935650 7854 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:46.935794 7854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:46.936206 7854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:46.939230 7854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "0d904d0a86e844de9d03ff6e7bb1a79e"
format_stamp: "Formatted at 2026-08-08 13:03:46 on dist-test-slave-t3q3"
server_key: "5b10f2c678f8283c79bd129641c45c58"
server_key_iv: "2a7e077128c4d21b4365da49f898fac6"
server_key_version: "encryptionkey@0"
I20260808 13:03:46.940171 7854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "0d904d0a86e844de9d03ff6e7bb1a79e"
format_stamp: "Formatted at 2026-08-08 13:03:46 on dist-test-slave-t3q3"
server_key: "5b10f2c678f8283c79bd129641c45c58"
server_key_iv: "2a7e077128c4d21b4365da49f898fac6"
server_key_version: "encryptionkey@0"
I20260808 13:03:46.945523 7854 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 13:03:46.949273 7872 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:46.950738 7854 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 13:03:46.950932 7854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "0d904d0a86e844de9d03ff6e7bb1a79e"
format_stamp: "Formatted at 2026-08-08 13:03:46 on dist-test-slave-t3q3"
server_key: "5b10f2c678f8283c79bd129641c45c58"
server_key_iv: "2a7e077128c4d21b4365da49f898fac6"
server_key_version: "encryptionkey@0"
I20260808 13:03:46.951180 7854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:46.976673 7854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:46.981353 7854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:46.981642 7854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:46.982642 7854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:03:46.983984 7854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:46.984146 7854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:46.984237 7854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:46.984295 7854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:47.017578 7854 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:41789
I20260808 13:03:47.017676 8047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:41789 every 8 connection(s)
I20260808 13:03:47.018488 7854 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:03:47.021660 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 7854
I20260808 13:03:47.021895 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:03:47.022564 1522 external_mini_cluster.cc:1428] Setting key 713ad8ec52d20216539738bc6bee7672
Aug 08 13:03:47 dist-test-slave-t3q3 krb5kdc[7689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194226, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:47.041669 8052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:47.021317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:38411 (local address 127.1.124.190:34977)
0808 13:03:47.021832 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:47.021838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:47.022888 (+ 1050us) server_negotiation.cc:407] Connection header received
0808 13:03:47.025060 (+ 2172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:47.025067 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:47.025155 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:47.025282 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:47.026737 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:47.027960 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:47.030331 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:47.031032 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:47.031185 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:47.035504 (+ 4319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:47.035509 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:47.035512 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:47.035544 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:47.038257 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:47.039250 (+ 993us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:47.039254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:47.039256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:47.039373 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:47.039864 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:47.039867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:47.039869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:47.040186 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:47.040298 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:47.041224 (+ 926us) server_negotiation.cc:299] Negotiation successful
0808 13:03:47.041396 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":272,"threads_started":1}
I20260808 13:03:47.043452 8053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:47.021823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:34977 (local address 127.1.124.129:38411)
0808 13:03:47.022716 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:47.022762 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:47.024794 (+ 2032us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:47.025467 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:47.025476 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:47.026011 (+ 535us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:47.026558 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:47.026574 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:47.028153 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:47.028158 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:47.030162 (+ 2004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:47.030179 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:47.031262 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:47.031270 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:47.031548 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:47.032368 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:47.032399 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:47.035273 (+ 2874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:47.038408 (+ 3135us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:47.038420 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:47.038440 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:47.039117 (+ 677us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:47.039483 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:47.039490 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:47.039496 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:47.039747 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:47.040317 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:47.040327 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:47.040811 (+ 484us) client_negotiation.cc:769] Sending connection context
0808 13:03:47.041132 (+ 321us) client_negotiation.cc:240] Negotiation successful
0808 13:03:47.041800 (+ 668us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":385,"threads_started":1}
I20260808 13:03:47.045264 8050 heartbeater.cc:347] Connected to a master server at 127.1.124.190:34977
I20260808 13:03:47.045610 8050 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:47.046335 8050 heartbeater.cc:511] Master 127.1.124.190:34977 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.1.124.130@KRBTEST.COM; defaulting to no policy
I20260808 13:03:47.048667 7760 ts_manager.cc:194] Registered new tserver with Master: 0d904d0a86e844de9d03ff6e7bb1a79e (127.1.124.129:41789)
I20260808 13:03:47.050304 7760 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:38411
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
I20260808 13:03:47.081712 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:34977
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:44949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:47.425303 8057 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:47.425591 8057 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:47.425696 8057 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:47.433099 8057 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:47.433223 8057 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:47.433342 8057 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:03:47.443688 8057 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:44949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:34977
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:47.445021 8057 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:47.446380 8057 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:47.460027 8065 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:47.460121 8063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:47.460139 8062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:47.465742 8057 server_base.cc:1034] running on GCE node
I20260808 13:03:47.466414 8057 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:47.467398 8057 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:47.468618 8057 hybrid_clock.cc:648] HybridClock initialized: now 1786194227468546 us; error 63 us; skew 500 ppm
I20260808 13:03:47.468953 8057 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:47 dist-test-slave-t3q3 krb5kdc[7689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194227, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:47.472304 8057 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:03:47.473692 8057 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:47.473788 8057 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:47.474077 8057 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:47.477277 8057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "72616d3eb2f640acb085a6883478e35f"
format_stamp: "Formatted at 2026-08-08 13:03:47 on dist-test-slave-t3q3"
server_key: "724872feab5d40897654b4e0404baf9e"
server_key_iv: "62f88bbe6c4e5dec30bbbf8b946531b2"
server_key_version: "encryptionkey@0"
I20260808 13:03:47.478094 8057 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "72616d3eb2f640acb085a6883478e35f"
format_stamp: "Formatted at 2026-08-08 13:03:47 on dist-test-slave-t3q3"
server_key: "724872feab5d40897654b4e0404baf9e"
server_key_iv: "62f88bbe6c4e5dec30bbbf8b946531b2"
server_key_version: "encryptionkey@0"
I20260808 13:03:47.483424 8057 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:03:47.487248 8075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:47.488910 8057 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:03:47.489049 8057 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "72616d3eb2f640acb085a6883478e35f"
format_stamp: "Formatted at 2026-08-08 13:03:47 on dist-test-slave-t3q3"
server_key: "724872feab5d40897654b4e0404baf9e"
server_key_iv: "62f88bbe6c4e5dec30bbbf8b946531b2"
server_key_version: "encryptionkey@0"
I20260808 13:03:47.489249 8057 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:47.508278 8057 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:47.512809 8057 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:47.513074 8057 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:47.514066 8057 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:03:47.515461 8057 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:47.515558 8057 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:47.515635 8057 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:47.515765 8057 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:47.551730 8057 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:35629
I20260808 13:03:47.551829 8257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:35629 every 8 connection(s)
I20260808 13:03:47.552691 8057 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:03:47.556879 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 8057
I20260808 13:03:47.557076 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:03:47.557544 1522 external_mini_cluster.cc:1428] Setting key 586258d481776aa35c7e9eca6a6185b4
Aug 08 13:03:47 dist-test-slave-t3q3 krb5kdc[7689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194227, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:47.573436 8260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:47.555212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:40717 (local address 127.1.124.190:34977)
0808 13:03:47.555724 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:47.555730 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:47.556840 (+ 1110us) server_negotiation.cc:407] Connection header received
0808 13:03:47.558860 (+ 2020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:47.558866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:47.558965 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:47.559089 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:47.560701 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:47.561448 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:47.563074 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:47.563506 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:47.563658 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:47.567463 (+ 3805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:47.567468 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:47.567471 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:47.567505 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:47.570122 (+ 2617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:47.570884 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:47.570889 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:47.570891 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:47.571013 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:47.571472 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:47.571476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:47.571478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:47.571749 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:47.571908 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:47.572810 (+ 902us) server_negotiation.cc:299] Negotiation successful
0808 13:03:47.573058 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":218,"threads_started":1}
I20260808 13:03:47.574482 8262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:47.555801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:34977 (local address 127.1.124.130:40717)
0808 13:03:47.556656 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:47.556725 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:47.558579 (+ 1854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:47.559336 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:47.559345 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:47.559878 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:47.560519 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:47.560537 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:47.561618 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:47.561622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:47.562899 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:47.562911 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:47.563742 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:47.563750 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:47.563979 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:47.564767 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:47.564789 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:47.567216 (+ 2427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:47.570277 (+ 3061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:47.570287 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:47.570304 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:47.570744 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:47.571128 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:47.571133 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:47.571151 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:47.571338 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:47.571892 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:47.571900 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:47.572433 (+ 533us) client_negotiation.cc:769] Sending connection context
0808 13:03:47.572715 (+ 282us) client_negotiation.cc:240] Negotiation successful
0808 13:03:47.573310 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":330,"threads_started":1}
I20260808 13:03:47.576026 8258 heartbeater.cc:347] Connected to a master server at 127.1.124.190:34977
I20260808 13:03:47.576407 8258 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:47.577095 8258 heartbeater.cc:511] Master 127.1.124.190:34977 requested a full tablet report, sending...
I20260808 13:03:47.578688 7760 ts_manager.cc:194] Registered new tserver with Master: 72616d3eb2f640acb085a6883478e35f (127.1.124.130:35629)
I20260808 13:03:47.579947 7760 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:40717
WARNING: no policy specified for kudu/127.1.124.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
I20260808 13:03:47.609298 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:34977
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:44949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:03:47.945878 8266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:47.946149 8266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:47.946249 8266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:47.953226 8266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:47.953332 8266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:47.953444 8266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:03:47.962947 8266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:44949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:34977
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:47.964198 8266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:47.965421 8266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:47.978327 8273 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:47.978375 8278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:47.978281 8276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:47.979516 8266 server_base.cc:1034] running on GCE node
I20260808 13:03:47.980268 8266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:47.981248 8266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:47.982452 8266 hybrid_clock.cc:648] HybridClock initialized: now 1786194227982397 us; error 45 us; skew 500 ppm
I20260808 13:03:47.982785 8266 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:03:47 dist-test-slave-t3q3 krb5kdc[7689](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194227, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:03:47.986282 8266 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:03:47.987649 8266 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:47.987753 8266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:47.988036 8266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:47.991137 8266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "a089a21d8cf544dfa4b3ea0fea3d7a29"
format_stamp: "Formatted at 2026-08-08 13:03:47 on dist-test-slave-t3q3"
server_key: "30570f6c049167b04cb2aaf1ee64ce84"
server_key_iv: "40d79527c07831db65930afbe602917d"
server_key_version: "encryptionkey@0"
I20260808 13:03:47.991971 8266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "a089a21d8cf544dfa4b3ea0fea3d7a29"
format_stamp: "Formatted at 2026-08-08 13:03:47 on dist-test-slave-t3q3"
server_key: "30570f6c049167b04cb2aaf1ee64ce84"
server_key_iv: "40d79527c07831db65930afbe602917d"
server_key_version: "encryptionkey@0"
I20260808 13:03:47.997617 8266 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260808 13:03:48.001591 8287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:48.003058 8266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:03:48.003257 8266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "a089a21d8cf544dfa4b3ea0fea3d7a29"
format_stamp: "Formatted at 2026-08-08 13:03:47 on dist-test-slave-t3q3"
server_key: "30570f6c049167b04cb2aaf1ee64ce84"
server_key_iv: "40d79527c07831db65930afbe602917d"
server_key_version: "encryptionkey@0"
I20260808 13:03:48.003499 8266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:48.022342 8266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:48.027583 8266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:48.027944 8266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:48.029161 8266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:03:48.030678 8266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:48.030786 8266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:48.030879 8266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:48.030939 8266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:48.053769 8050 heartbeater.cc:503] Master 127.1.124.190:34977 was elected leader, sending a full tablet report...
I20260808 13:03:48.068440 8266 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:37193
I20260808 13:03:48.068498 8471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:37193 every 8 connection(s)
I20260808 13:03:48.069352 8266 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:03:48.072463 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 8266
I20260808 13:03:48.072670 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:03:48.073324 1522 external_mini_cluster.cc:1428] Setting key 1a7d25462ebb4d9a669880dbc44ee4ae
Aug 08 13:03:48 dist-test-slave-t3q3 krb5kdc[7689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194227, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:03:48.090391 8260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:48.071881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:43741 (local address 127.1.124.190:34977)
0808 13:03:48.072128 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:48.072133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:48.073481 (+ 1348us) server_negotiation.cc:407] Connection header received
0808 13:03:48.075748 (+ 2267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:48.075753 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:48.075838 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:48.075922 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:48.077694 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.078474 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.080033 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.080506 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.080624 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:48.083996 (+ 3372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:48.084001 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:48.084004 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:48.084132 (+ 128us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:48.086917 (+ 2785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:48.087851 (+ 934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:48.087856 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:48.087858 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:48.088108 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:48.088576 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:48.088580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:48.088582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:48.089022 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:48.089186 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:48.089967 (+ 781us) server_negotiation.cc:299] Negotiation successful
0808 13:03:48.090164 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 13:03:48.091429 8475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:48.072483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:34977 (local address 127.1.124.131:43741)
0808 13:03:48.073290 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:48.073338 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:48.075433 (+ 2095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:48.076238 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:48.076249 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:48.076960 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:48.077534 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:48.077555 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.078609 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.078612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:48.079852 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:48.079864 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.080708 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.080713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:48.080904 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:48.081542 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:03:48.081564 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:48.083781 (+ 2217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:48.087085 (+ 3304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:48.087092 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:48.087106 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:48.087694 (+ 588us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:48.088239 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:03:48.088243 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:03:48.088245 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:03:48.088425 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:03:48.089195 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:48.089206 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:48.089635 (+ 429us) client_negotiation.cc:769] Sending connection context
0808 13:03:48.089882 (+ 247us) client_negotiation.cc:240] Negotiation successful
0808 13:03:48.090298 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":352,"threads_started":1}
I20260808 13:03:48.092914 8473 heartbeater.cc:347] Connected to a master server at 127.1.124.190:34977
I20260808 13:03:48.093273 8473 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:48.093972 8473 heartbeater.cc:511] Master 127.1.124.190:34977 requested a full tablet report, sending...
I20260808 13:03:48.095407 7760 ts_manager.cc:194] Registered new tserver with Master: a089a21d8cf544dfa4b3ea0fea3d7a29 (127.1.124.131:37193)
I20260808 13:03:48.096338 7760 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:43741
I20260808 13:03:48.101557 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:03:48.119087 8260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:48.106080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34736 (local address 127.1.124.190:34977)
0808 13:03:48.106279 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:48.106287 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:48.106744 (+ 457us) server_negotiation.cc:407] Connection header received
0808 13:03:48.106945 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:48.106950 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:48.107027 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:48.107142 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:48.108697 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.109426 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.110720 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.111178 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.111318 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:48.113634 (+ 2316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:48.113640 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:48.113645 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:48.113686 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:48.116389 (+ 2703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:48.117070 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:48.117076 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:48.117080 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:48.117228 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:48.117620 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:48.117626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:48.117631 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:48.117955 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:48.118243 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:48.118535 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 13:03:48.118766 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 13:03:48.123296 7760 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34736:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:03:48.130344 7760 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:03:48.161566 8489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:48.143402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:35629 (local address 127.0.0.1:44510)
0808 13:03:48.145345 (+ 1943us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:48.145409 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:48.145754 (+ 345us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:48.148571 (+ 2817us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:48.148582 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:48.148647 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:48.149169 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:48.149185 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.153380 (+ 4195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.153389 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:48.158447 (+ 5058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:48.158471 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.160712 (+ 2241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.160718 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:48.160934 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:48.161001 (+ 67us) client_negotiation.cc:769] Sending connection context
0808 13:03:48.161123 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 13:03:48.161301 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1646,"thread_start_us":1491,"threads_started":1}
I20260808 13:03:48.162422 8490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:48.143608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44510 (local address 127.1.124.130:35629)
0808 13:03:48.144739 (+ 1131us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:48.144746 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:48.145546 (+ 800us) server_negotiation.cc:407] Connection header received
0808 13:03:48.146061 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:48.146082 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:48.147883 (+ 1801us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:48.148154 (+ 271us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:48.149413 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.153118 (+ 3705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.158682 (+ 5564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.160198 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.160353 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:48.160568 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:48.162055 (+ 1487us) server_negotiation.cc:299] Negotiation successful
0808 13:03:48.162193 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":955,"thread_start_us":842,"threads_started":1}
I20260808 13:03:48.166286 8147 tablet_service.cc:1467] Processing CreateTablet for tablet 0639dac797a94eeea61393de32f28724 (DEFAULT_TABLE table=test-table [id=53976d0f642a431cb5a6a66f1c86b67b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:48.167053 8494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:48.146571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:37193 (local address 127.0.0.1:55272)
0808 13:03:48.157602 (+ 11031us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:48.157631 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:48.157774 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:48.159139 (+ 1365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:48.159143 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:48.159170 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:48.159627 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:48.159640 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.161781 (+ 2141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.161789 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:48.164285 (+ 2496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:48.164294 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.166094 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.166101 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:48.166274 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:48.166320 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 13:03:48.166407 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 13:03:48.166586 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10875,"mutex_wait_us":94,"thread_start_us":10714,"threads_started":1}
I20260808 13:03:48.167234 8493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:48.146231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55272 (local address 127.1.124.131:37193)
0808 13:03:48.158406 (+ 12175us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:48.158419 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:48.158461 (+ 42us) server_negotiation.cc:407] Connection header received
0808 13:03:48.158572 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:48.158583 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:48.158878 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:48.159117 (+ 239us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:48.159900 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.161598 (+ 1698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.164448 (+ 2850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.165852 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.165987 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:48.166209 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:48.166638 (+ 429us) server_negotiation.cc:299] Negotiation successful
0808 13:03:48.166833 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11987,"thread_start_us":11826,"threads_started":1}
I20260808 13:03:48.168376 8147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0639dac797a94eeea61393de32f28724. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:48.168905 8491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:48.144402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:41789 (local address 127.0.0.1:60426)
0808 13:03:48.156554 (+ 12152us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:48.156579 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:48.156731 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:48.159440 (+ 2709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:48.159446 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:48.159483 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:48.159935 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:48.159950 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.161899 (+ 1949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.161905 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:48.164275 (+ 2370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:48.164292 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.168088 (+ 3796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.168096 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:48.168276 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:48.168316 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 13:03:48.168413 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 13:03:48.168547 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11980,"mutex_wait_us":34,"thread_start_us":12427,"threads_started":1}
I20260808 13:03:48.168991 8492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:48.145066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60426 (local address 127.1.124.129:41789)
0808 13:03:48.158797 (+ 13731us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:48.158805 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:48.158834 (+ 29us) server_negotiation.cc:407] Connection header received
0808 13:03:48.158931 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:48.158940 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:48.159168 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:48.159366 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:48.160209 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.161736 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.166311 (+ 4575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.167748 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.167905 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:48.168164 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:48.168506 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 13:03:48.168683 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13560,"thread_start_us":13397,"threads_started":1}
I20260808 13:03:48.170779 8365 tablet_service.cc:1467] Processing CreateTablet for tablet 0639dac797a94eeea61393de32f28724 (DEFAULT_TABLE table=test-table [id=53976d0f642a431cb5a6a66f1c86b67b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:48.172331 8365 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0639dac797a94eeea61393de32f28724. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:48.172626 7944 tablet_service.cc:1467] Processing CreateTablet for tablet 0639dac797a94eeea61393de32f28724 (DEFAULT_TABLE table=test-table [id=53976d0f642a431cb5a6a66f1c86b67b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:48.173982 7944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0639dac797a94eeea61393de32f28724. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:48.181046 8495 tablet_bootstrap.cc:492] T 0639dac797a94eeea61393de32f28724 P 72616d3eb2f640acb085a6883478e35f: Bootstrap starting.
I20260808 13:03:48.184728 8495 tablet_bootstrap.cc:654] T 0639dac797a94eeea61393de32f28724 P 72616d3eb2f640acb085a6883478e35f: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:48.185765 8495 log.cc:824] T 0639dac797a94eeea61393de32f28724 P 72616d3eb2f640acb085a6883478e35f: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:48.188514 8495 tablet_bootstrap.cc:492] T 0639dac797a94eeea61393de32f28724 P 72616d3eb2f640acb085a6883478e35f: No bootstrap required, opened a new log
I20260808 13:03:48.188812 8495 ts_tablet_manager.cc:1397] T 0639dac797a94eeea61393de32f28724 P 72616d3eb2f640acb085a6883478e35f: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260808 13:03:48.188997 8497 tablet_bootstrap.cc:492] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29: Bootstrap starting.
I20260808 13:03:48.191727 8499 tablet_bootstrap.cc:492] T 0639dac797a94eeea61393de32f28724 P 0d904d0a86e844de9d03ff6e7bb1a79e: Bootstrap starting.
I20260808 13:03:48.193408 8497 tablet_bootstrap.cc:654] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:48.195813 8499 tablet_bootstrap.cc:654] T 0639dac797a94eeea61393de32f28724 P 0d904d0a86e844de9d03ff6e7bb1a79e: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:48.196712 8497 log.cc:824] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:48.197395 8499 log.cc:824] T 0639dac797a94eeea61393de32f28724 P 0d904d0a86e844de9d03ff6e7bb1a79e: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:48.200999 8497 tablet_bootstrap.cc:492] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29: No bootstrap required, opened a new log
I20260808 13:03:48.201356 8497 ts_tablet_manager.cc:1397] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.001s
I20260808 13:03:48.203590 8495 raft_consensus.cc:348] T 0639dac797a94eeea61393de32f28724 P 72616d3eb2f640acb085a6883478e35f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72616d3eb2f640acb085a6883478e35f" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 35629 } } peers { permanent_uuid: "a089a21d8cf544dfa4b3ea0fea3d7a29" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37193 } } peers { permanent_uuid: "0d904d0a86e844de9d03ff6e7bb1a79e" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41789 } }
I20260808 13:03:48.204166 8495 raft_consensus.cc:374] T 0639dac797a94eeea61393de32f28724 P 72616d3eb2f640acb085a6883478e35f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:48.204299 8495 raft_consensus.cc:729] T 0639dac797a94eeea61393de32f28724 P 72616d3eb2f640acb085a6883478e35f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72616d3eb2f640acb085a6883478e35f, State: Initialized, Role: FOLLOWER
I20260808 13:03:48.205039 8495 consensus_queue.cc:260] T 0639dac797a94eeea61393de32f28724 P 72616d3eb2f640acb085a6883478e35f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72616d3eb2f640acb085a6883478e35f" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 35629 } } peers { permanent_uuid: "a089a21d8cf544dfa4b3ea0fea3d7a29" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37193 } } peers { permanent_uuid: "0d904d0a86e844de9d03ff6e7bb1a79e" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41789 } }
I20260808 13:03:48.206391 8258 heartbeater.cc:503] Master 127.1.124.190:34977 was elected leader, sending a full tablet report...
I20260808 13:03:48.209192 8495 ts_tablet_manager.cc:1428] T 0639dac797a94eeea61393de32f28724 P 72616d3eb2f640acb085a6883478e35f: Time spent starting tablet: real 0.020s user 0.016s sys 0.000s
I20260808 13:03:48.209861 8499 tablet_bootstrap.cc:492] T 0639dac797a94eeea61393de32f28724 P 0d904d0a86e844de9d03ff6e7bb1a79e: No bootstrap required, opened a new log
I20260808 13:03:48.210237 8499 ts_tablet_manager.cc:1397] T 0639dac797a94eeea61393de32f28724 P 0d904d0a86e844de9d03ff6e7bb1a79e: Time spent bootstrapping tablet: real 0.019s user 0.007s sys 0.000s
I20260808 13:03:48.219520 8497 raft_consensus.cc:348] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72616d3eb2f640acb085a6883478e35f" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 35629 } } peers { permanent_uuid: "a089a21d8cf544dfa4b3ea0fea3d7a29" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37193 } } peers { permanent_uuid: "0d904d0a86e844de9d03ff6e7bb1a79e" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41789 } }
I20260808 13:03:48.219969 8497 raft_consensus.cc:374] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:48.220114 8497 raft_consensus.cc:729] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a089a21d8cf544dfa4b3ea0fea3d7a29, State: Initialized, Role: FOLLOWER
I20260808 13:03:48.220788 8497 consensus_queue.cc:260] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72616d3eb2f640acb085a6883478e35f" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 35629 } } peers { permanent_uuid: "a089a21d8cf544dfa4b3ea0fea3d7a29" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37193 } } peers { permanent_uuid: "0d904d0a86e844de9d03ff6e7bb1a79e" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41789 } }
I20260808 13:03:48.221144 8499 raft_consensus.cc:348] T 0639dac797a94eeea61393de32f28724 P 0d904d0a86e844de9d03ff6e7bb1a79e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72616d3eb2f640acb085a6883478e35f" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 35629 } } peers { permanent_uuid: "a089a21d8cf544dfa4b3ea0fea3d7a29" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37193 } } peers { permanent_uuid: "0d904d0a86e844de9d03ff6e7bb1a79e" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41789 } }
I20260808 13:03:48.221576 8499 raft_consensus.cc:374] T 0639dac797a94eeea61393de32f28724 P 0d904d0a86e844de9d03ff6e7bb1a79e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:48.221714 8499 raft_consensus.cc:729] T 0639dac797a94eeea61393de32f28724 P 0d904d0a86e844de9d03ff6e7bb1a79e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d904d0a86e844de9d03ff6e7bb1a79e, State: Initialized, Role: FOLLOWER
I20260808 13:03:48.222123 8473 heartbeater.cc:503] Master 127.1.124.190:34977 was elected leader, sending a full tablet report...
I20260808 13:03:48.222460 8499 consensus_queue.cc:260] T 0639dac797a94eeea61393de32f28724 P 0d904d0a86e844de9d03ff6e7bb1a79e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72616d3eb2f640acb085a6883478e35f" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 35629 } } peers { permanent_uuid: "a089a21d8cf544dfa4b3ea0fea3d7a29" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37193 } } peers { permanent_uuid: "0d904d0a86e844de9d03ff6e7bb1a79e" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41789 } }
I20260808 13:03:48.222772 8497 ts_tablet_manager.cc:1428] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29: Time spent starting tablet: real 0.021s user 0.011s sys 0.003s
I20260808 13:03:48.224468 8499 ts_tablet_manager.cc:1428] T 0639dac797a94eeea61393de32f28724 P 0d904d0a86e844de9d03ff6e7bb1a79e: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
W20260808 13:03:48.270983 8051 tablet.cc:2369] T 0639dac797a94eeea61393de32f28724 P 0d904d0a86e844de9d03ff6e7bb1a79e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:03:48.304837 8259 tablet.cc:2369] T 0639dac797a94eeea61393de32f28724 P 72616d3eb2f640acb085a6883478e35f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:03:48.313407 8505 raft_consensus.cc:482] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:03:48.313773 8505 raft_consensus.cc:504] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72616d3eb2f640acb085a6883478e35f" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 35629 } } peers { permanent_uuid: "a089a21d8cf544dfa4b3ea0fea3d7a29" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37193 } } peers { permanent_uuid: "0d904d0a86e844de9d03ff6e7bb1a79e" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41789 } }
I20260808 13:03:48.315537 8505 leader_election.cc:288] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 72616d3eb2f640acb085a6883478e35f (127.1.124.130:35629), 0d904d0a86e844de9d03ff6e7bb1a79e (127.1.124.129:41789)
W20260808 13:03:48.320880 8474 tablet.cc:2369] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:03:48.322590 8490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:48.316307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:40963 (local address 127.1.124.130:35629)
0808 13:03:48.316486 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:48.316493 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:48.316520 (+ 27us) server_negotiation.cc:407] Connection header received
0808 13:03:48.316783 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:48.316790 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:48.316877 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:48.316990 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:48.317551 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.318492 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.320703 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.321692 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.321860 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:48.321938 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:48.322206 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 13:03:48.322333 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 13:03:48.322602 8475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:48.315969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:35629 (local address 127.1.124.131:40963)
0808 13:03:48.316360 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:48.316387 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:48.316587 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:48.317024 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:48.317029 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:48.317168 (+ 139us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:48.317388 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:48.317405 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.318648 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.318654 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:48.320475 (+ 1821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:48.320488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.321852 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.321858 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:48.322039 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:48.322063 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 13:03:48.322167 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 13:03:48.322299 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"spinlock_wait_cycles":25472}
I20260808 13:03:48.323346 8492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:48.316251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:50065 (local address 127.1.124.129:41789)
0808 13:03:48.316460 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:48.316466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:48.317137 (+ 671us) server_negotiation.cc:407] Connection header received
0808 13:03:48.317251 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:48.317257 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:48.317344 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:48.317442 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:03:48.318117 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.319182 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.321219 (+ 2037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.322312 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.322517 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:48.322606 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:48.322993 (+ 387us) server_negotiation.cc:299] Negotiation successful
0808 13:03:48.323093 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 13:03:48.323573 8508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:48.316130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:41789 (local address 127.1.124.131:50065)
0808 13:03:48.316971 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:48.316990 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:48.317119 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:48.317488 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:48.317494 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:48.317519 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:03:48.317896 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:48.317909 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.319349 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.319353 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:48.321056 (+ 1703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:48.321071 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.322542 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.322549 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:48.322778 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:48.322834 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 13:03:48.322947 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 13:03:48.323113 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":709,"mutex_wait_us":4,"thread_start_us":495,"threads_started":1}
I20260808 13:03:48.324242 8185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0639dac797a94eeea61393de32f28724" candidate_uuid: "a089a21d8cf544dfa4b3ea0fea3d7a29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72616d3eb2f640acb085a6883478e35f" is_pre_election: true
I20260808 13:03:48.324774 8185 raft_consensus.cc:2454] T 0639dac797a94eeea61393de32f28724 P 72616d3eb2f640acb085a6883478e35f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a089a21d8cf544dfa4b3ea0fea3d7a29 in term 0.
I20260808 13:03:48.324822 7983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0639dac797a94eeea61393de32f28724" candidate_uuid: "a089a21d8cf544dfa4b3ea0fea3d7a29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d904d0a86e844de9d03ff6e7bb1a79e" is_pre_election: true
I20260808 13:03:48.325373 7983 raft_consensus.cc:2454] T 0639dac797a94eeea61393de32f28724 P 0d904d0a86e844de9d03ff6e7bb1a79e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a089a21d8cf544dfa4b3ea0fea3d7a29 in term 0.
I20260808 13:03:48.325452 8290 leader_election.cc:302] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72616d3eb2f640acb085a6883478e35f, a089a21d8cf544dfa4b3ea0fea3d7a29; no voters:
I20260808 13:03:48.325834 8505 raft_consensus.cc:2781] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:03:48.325994 8505 raft_consensus.cc:482] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:03:48.326118 8505 raft_consensus.cc:3037] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:48.328485 8505 raft_consensus.cc:504] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72616d3eb2f640acb085a6883478e35f" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 35629 } } peers { permanent_uuid: "a089a21d8cf544dfa4b3ea0fea3d7a29" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37193 } } peers { permanent_uuid: "0d904d0a86e844de9d03ff6e7bb1a79e" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41789 } }
I20260808 13:03:48.329149 8505 leader_election.cc:288] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29 [CANDIDATE]: Term 1 election: Requested vote from peers 72616d3eb2f640acb085a6883478e35f (127.1.124.130:35629), 0d904d0a86e844de9d03ff6e7bb1a79e (127.1.124.129:41789)
I20260808 13:03:48.329574 8185 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0639dac797a94eeea61393de32f28724" candidate_uuid: "a089a21d8cf544dfa4b3ea0fea3d7a29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72616d3eb2f640acb085a6883478e35f"
I20260808 13:03:48.329697 7983 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0639dac797a94eeea61393de32f28724" candidate_uuid: "a089a21d8cf544dfa4b3ea0fea3d7a29" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d904d0a86e844de9d03ff6e7bb1a79e"
I20260808 13:03:48.329790 8185 raft_consensus.cc:3037] T 0639dac797a94eeea61393de32f28724 P 72616d3eb2f640acb085a6883478e35f [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:48.329947 7983 raft_consensus.cc:3037] T 0639dac797a94eeea61393de32f28724 P 0d904d0a86e844de9d03ff6e7bb1a79e [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:48.332034 8185 raft_consensus.cc:2454] T 0639dac797a94eeea61393de32f28724 P 72616d3eb2f640acb085a6883478e35f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a089a21d8cf544dfa4b3ea0fea3d7a29 in term 1.
I20260808 13:03:48.332326 7983 raft_consensus.cc:2454] T 0639dac797a94eeea61393de32f28724 P 0d904d0a86e844de9d03ff6e7bb1a79e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a089a21d8cf544dfa4b3ea0fea3d7a29 in term 1.
I20260808 13:03:48.332515 8290 leader_election.cc:302] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72616d3eb2f640acb085a6883478e35f, a089a21d8cf544dfa4b3ea0fea3d7a29; no voters:
I20260808 13:03:48.332831 8505 raft_consensus.cc:2781] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:48.333288 8505 raft_consensus.cc:686] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29 [term 1 LEADER]: Becoming Leader. State: Replica: a089a21d8cf544dfa4b3ea0fea3d7a29, State: Running, Role: LEADER
I20260808 13:03:48.333791 8505 consensus_queue.cc:237] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72616d3eb2f640acb085a6883478e35f" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 35629 } } peers { permanent_uuid: "a089a21d8cf544dfa4b3ea0fea3d7a29" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37193 } } peers { permanent_uuid: "0d904d0a86e844de9d03ff6e7bb1a79e" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41789 } }
I20260808 13:03:48.338455 7757 catalog_manager.cc:5495] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29 reported cstate change: term changed from 0 to 1, leader changed from <none> to a089a21d8cf544dfa4b3ea0fea3d7a29 (127.1.124.131). New cstate: current_term: 1 leader_uuid: "a089a21d8cf544dfa4b3ea0fea3d7a29" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "72616d3eb2f640acb085a6883478e35f" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 35629 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a089a21d8cf544dfa4b3ea0fea3d7a29" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37193 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0d904d0a86e844de9d03ff6e7bb1a79e" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41789 } health_report { overall_health: UNKNOWN } } }
I20260808 13:03:48.397549 8493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:48.391144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55280 (local address 127.1.124.131:37193)
0808 13:03:48.391336 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:48.391340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:48.391358 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:03:48.391514 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:48.391518 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:48.391597 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:48.391733 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:03:48.392180 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.393131 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.395064 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.395471 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.395572 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:48.396276 (+ 704us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:03:48.396931 (+ 655us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:03:48.397082 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:48.397200 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 13:03:48.397277 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 13:03:48.407841 8185 raft_consensus.cc:1261] T 0639dac797a94eeea61393de32f28724 P 72616d3eb2f640acb085a6883478e35f [term 1 FOLLOWER]: Refusing update from remote peer a089a21d8cf544dfa4b3ea0fea3d7a29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:03:48.407832 7983 raft_consensus.cc:1261] T 0639dac797a94eeea61393de32f28724 P 0d904d0a86e844de9d03ff6e7bb1a79e [term 1 FOLLOWER]: Refusing update from remote peer a089a21d8cf544dfa4b3ea0fea3d7a29: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:03:48.409052 8510 consensus_queue.cc:1035] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d904d0a86e844de9d03ff6e7bb1a79e" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41789 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:03:48.409425 8505 consensus_queue.cc:1035] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29 [LEADER]: Connected to new peer: Peer: permanent_uuid: "72616d3eb2f640acb085a6883478e35f" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 35629 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:03:48.437363 7760 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:03:48.437685 7760 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:03:48.441535 7760 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 63f435c0b1b94dfb9a2ecbdeef3c11cf: Sending DeleteTablet for 3 replicas of tablet 0639dac797a94eeea61393de32f28724
I20260808 13:03:48.442763 8147 tablet_service.cc:1514] Processing DeleteTablet for tablet 0639dac797a94eeea61393de32f28724 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:48 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:44510
I20260808 13:03:48.442763 8365 tablet_service.cc:1514] Processing DeleteTablet for tablet 0639dac797a94eeea61393de32f28724 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:48 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:55272
I20260808 13:03:48.442864 7944 tablet_service.cc:1514] Processing DeleteTablet for tablet 0639dac797a94eeea61393de32f28724 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:48 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:60426
I20260808 13:03:48.445703 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 7854
I20260808 13:03:48.475724 8538 tablet_replica.cc:330] T 0639dac797a94eeea61393de32f28724 P 72616d3eb2f640acb085a6883478e35f: stopping tablet replica
I20260808 13:03:48.476449 8538 raft_consensus.cc:2229] T 0639dac797a94eeea61393de32f28724 P 72616d3eb2f640acb085a6883478e35f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:03:48.476851 8537 tablet_replica.cc:330] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29: stopping tablet replica
I20260808 13:03:48.477327 8537 raft_consensus.cc:2229] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:03:48.478055 8537 raft_consensus.cc:2258] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:48.478875 8539 tablet_replica.cc:330] T 0639dac797a94eeea61393de32f28724 P 0d904d0a86e844de9d03ff6e7bb1a79e: stopping tablet replica
I20260808 13:03:48.479331 8539 raft_consensus.cc:2229] T 0639dac797a94eeea61393de32f28724 P 0d904d0a86e844de9d03ff6e7bb1a79e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:03:48.479892 8539 raft_consensus.cc:2258] T 0639dac797a94eeea61393de32f28724 P 0d904d0a86e844de9d03ff6e7bb1a79e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:48.479465 8538 raft_consensus.cc:2258] T 0639dac797a94eeea61393de32f28724 P 72616d3eb2f640acb085a6883478e35f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:48.480937 8537 ts_tablet_manager.cc:1905] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:48.482000 8539 ts_tablet_manager.cc:1905] T 0639dac797a94eeea61393de32f28724 P 0d904d0a86e844de9d03ff6e7bb1a79e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:48.484714 8538 ts_tablet_manager.cc:1905] T 0639dac797a94eeea61393de32f28724 P 72616d3eb2f640acb085a6883478e35f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:48.489389 8539 ts_tablet_manager.cc:1918] T 0639dac797a94eeea61393de32f28724 P 0d904d0a86e844de9d03ff6e7bb1a79e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:03:48.489562 8537 ts_tablet_manager.cc:1918] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:03:48.489697 8539 log.cc:1196] T 0639dac797a94eeea61393de32f28724 P 0d904d0a86e844de9d03ff6e7bb1a79e: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/0639dac797a94eeea61393de32f28724
I20260808 13:03:48.489760 8537 log.cc:1196] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/0639dac797a94eeea61393de32f28724
I20260808 13:03:48.490356 8539 ts_tablet_manager.cc:1939] T 0639dac797a94eeea61393de32f28724 P 0d904d0a86e844de9d03ff6e7bb1a79e: Deleting consensus metadata
I20260808 13:03:48.490376 8537 ts_tablet_manager.cc:1939] T 0639dac797a94eeea61393de32f28724 P a089a21d8cf544dfa4b3ea0fea3d7a29: Deleting consensus metadata
I20260808 13:03:48.492547 7734 catalog_manager.cc:4841] TS a089a21d8cf544dfa4b3ea0fea3d7a29 (127.1.124.131:37193): tablet 0639dac797a94eeea61393de32f28724 (table test-table [id=53976d0f642a431cb5a6a66f1c86b67b]) successfully deleted
Aug 08 13:03:48 dist-test-slave-t3q3 krb5kdc[7689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.129@KRBTEST.COM
I20260808 13:03:48.492988 7731 catalog_manager.cc:4841] TS 0d904d0a86e844de9d03ff6e7bb1a79e (127.1.124.129:41789): tablet 0639dac797a94eeea61393de32f28724 (table test-table [id=53976d0f642a431cb5a6a66f1c86b67b]) successfully deleted
I20260808 13:03:48.495827 8538 ts_tablet_manager.cc:1918] T 0639dac797a94eeea61393de32f28724 P 72616d3eb2f640acb085a6883478e35f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:03:48.496042 8538 log.cc:1196] T 0639dac797a94eeea61393de32f28724 P 72616d3eb2f640acb085a6883478e35f: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/0639dac797a94eeea61393de32f28724
I20260808 13:03:48.496747 8538 ts_tablet_manager.cc:1939] T 0639dac797a94eeea61393de32f28724 P 72616d3eb2f640acb085a6883478e35f: Deleting consensus metadata
I20260808 13:03:48.498427 7734 catalog_manager.cc:4841] TS 72616d3eb2f640acb085a6883478e35f (127.1.124.130:35629): tablet 0639dac797a94eeea61393de32f28724 (table test-table [id=53976d0f642a431cb5a6a66f1c86b67b]) successfully deleted
I20260808 13:03:48.502403 8492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:48.447690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60436 (local address 127.1.124.129:41789)
0808 13:03:48.447922 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:48.447932 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:48.471750 (+ 23818us) server_negotiation.cc:407] Connection header received
0808 13:03:48.472434 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:48.472444 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:48.472563 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:48.472671 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:48.476796 (+ 4125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.477981 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.482250 (+ 4269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.482918 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.483035 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:48.494417 (+ 11382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:48.494432 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:48.494453 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:48.494501 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:48.498674 (+ 4173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:48.499767 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:48.499776 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:48.499778 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:48.499943 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:48.500469 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:48.500474 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:48.500478 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:48.501036 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:48.501305 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:48.501662 (+ 357us) server_negotiation.cc:299] Negotiation successful
0808 13:03:48.501971 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 13:03:48.503201 8042 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:48.594430 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 7854
I20260808 13:03:48.620849 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 8057
Aug 08 13:03:48 dist-test-slave-t3q3 krb5kdc[7689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.130@KRBTEST.COM
I20260808 13:03:48.640250 8490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:48.621786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44514 (local address 127.1.124.130:35629)
0808 13:03:48.621971 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:48.621976 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:48.621995 (+ 19us) server_negotiation.cc:407] Connection header received
0808 13:03:48.622209 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:48.622214 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:48.622286 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:48.622480 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:48.624129 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.624901 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.626838 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.627270 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.627437 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:48.632807 (+ 5370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:48.632831 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:48.632855 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:48.632913 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:48.637149 (+ 4236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:48.637913 (+ 764us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:48.637924 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:48.637928 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:48.638114 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:48.638478 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:48.638484 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:48.638487 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:48.638996 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:48.639318 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:48.639581 (+ 263us) server_negotiation.cc:299] Negotiation successful
0808 13:03:48.639827 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 13:03:48.641155 8252 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:48.741257 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 8057
I20260808 13:03:48.770764 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 8266
Aug 08 13:03:48 dist-test-slave-t3q3 krb5kdc[7689](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194225, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.131@KRBTEST.COM
I20260808 13:03:48.789641 8493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:48.771628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55288 (local address 127.1.124.131:37193)
0808 13:03:48.771809 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:48.771813 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:48.771884 (+ 71us) server_negotiation.cc:407] Connection header received
0808 13:03:48.772169 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:48.772173 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:48.772272 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:48.772359 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:48.774190 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.774933 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.776831 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:48.777337 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:48.777454 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:48.783026 (+ 5572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:48.783046 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:48.783061 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:03:48.783104 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:48.786374 (+ 3270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:48.787184 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:48.787196 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:48.787198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:48.787370 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:03:48.787792 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:03:48.787797 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:03:48.787803 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:03:48.788361 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:48.788743 (+ 382us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:48.789029 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 13:03:48.789258 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 13:03:48.790405 8460 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:48.918185 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 8266
I20260808 13:03:48.945689 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 7706
I20260808 13:03:48.946947 7809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:49.057195 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 7706
2026-08-08T13:03:49Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4682 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T13:03:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:03:49Z Disabled control of system clock
I20260808 13:03:49.145006 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:37121
--webserver_interface=127.1.124.190
--webserver_port=0
--builtin_ntp_servers=127.1.124.148:33341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:37121
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 13:03:49.485814 8555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:49.486117 8555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:49.486203 8555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:49.493654 8555 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:03:49.493777 8555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:49.493815 8555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:49.493878 8555 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:03:49.493930 8555 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:03:49.503571 8555 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:33341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:37121
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:37121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:49.504914 8555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:49.506718 8555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:49.519181 8566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:49.519116 8567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:49.519650 8574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:49.520141 8555 server_base.cc:1034] running on GCE node
I20260808 13:03:49.520929 8555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:49.522365 8555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:49.523578 8555 hybrid_clock.cc:648] HybridClock initialized: now 1786194229523539 us; error 35 us; skew 500 ppm
I20260808 13:03:49.523979 8555 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:03:49.526382 8555 webserver.cc:460] Webserver started at http://127.1.124.190:37859/ using document root <none> and password file <none>
I20260808 13:03:49.527094 8555 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:49.527218 8555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:49.527590 8555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:49.530630 8555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "c38a240490cc4067b7a8a48be24032d2"
format_stamp: "Formatted at 2026-08-08 13:03:49 on dist-test-slave-t3q3"
server_key: "370aa701082dd62393cdf7704c66885c"
server_key_iv: "47346929ece712b346a2e01812275b5d"
server_key_version: "encryptionkey@0"
I20260808 13:03:49.531451 8555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "c38a240490cc4067b7a8a48be24032d2"
format_stamp: "Formatted at 2026-08-08 13:03:49 on dist-test-slave-t3q3"
server_key: "370aa701082dd62393cdf7704c66885c"
server_key_iv: "47346929ece712b346a2e01812275b5d"
server_key_version: "encryptionkey@0"
I20260808 13:03:49.536690 8555 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 13:03:49.540201 8584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:49.541654 8555 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260808 13:03:49.541872 8555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "c38a240490cc4067b7a8a48be24032d2"
format_stamp: "Formatted at 2026-08-08 13:03:49 on dist-test-slave-t3q3"
server_key: "370aa701082dd62393cdf7704c66885c"
server_key_iv: "47346929ece712b346a2e01812275b5d"
server_key_version: "encryptionkey@0"
I20260808 13:03:49.542112 8555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:49.548139 8555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:49.548904 8555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:49.549219 8555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:49.568509 8555 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:37121
I20260808 13:03:49.568547 8674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:37121 every 8 connection(s)
I20260808 13:03:49.570009 8555 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:03:49.573937 8675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:49.578859 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 8555
I20260808 13:03:49.579228 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:03:49.579914 1522 external_mini_cluster.cc:1428] Setting key 1d208d2b2207fc09b9e7dd5a664ca276
I20260808 13:03:49.585498 8675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2: Bootstrap starting.
I20260808 13:03:49.589013 8681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:49.582190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41558 (local address 127.1.124.190:37121)
0808 13:03:49.583292 (+ 1102us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:49.583304 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:49.583353 (+ 49us) server_negotiation.cc:407] Connection header received
0808 13:03:49.585133 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:49.585160 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:49.585592 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:49.586055 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:49.587226 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:49.587233 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:49.587246 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:03:49.587259 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:49.587327 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:03:49.587343 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:49.587600 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:49.587745 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 13:03:49.588015 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":390,"threads_started":1}
I20260808 13:03:49.589053 8675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:49.590461 8675 log.cc:824] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:49.593492 8675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2: No bootstrap required, opened a new log
I20260808 13:03:49.601387 8675 raft_consensus.cc:348] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38a240490cc4067b7a8a48be24032d2" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37121 } }
I20260808 13:03:49.601814 8675 raft_consensus.cc:374] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:49.601934 8675 raft_consensus.cc:729] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c38a240490cc4067b7a8a48be24032d2, State: Initialized, Role: FOLLOWER
I20260808 13:03:49.602617 8675 consensus_queue.cc:260] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38a240490cc4067b7a8a48be24032d2" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37121 } }
I20260808 13:03:49.602954 8675 raft_consensus.cc:388] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:03:49.603107 8675 raft_consensus.cc:482] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:03:49.603307 8675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:49.605648 8675 raft_consensus.cc:504] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38a240490cc4067b7a8a48be24032d2" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37121 } }
I20260808 13:03:49.606215 8675 leader_election.cc:302] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c38a240490cc4067b7a8a48be24032d2; no voters:
I20260808 13:03:49.606827 8675 leader_election.cc:288] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:03:49.606981 8684 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:49.608142 8684 raft_consensus.cc:686] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2 [term 1 LEADER]: Becoming Leader. State: Replica: c38a240490cc4067b7a8a48be24032d2, State: Running, Role: LEADER
I20260808 13:03:49.608850 8684 consensus_queue.cc:237] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38a240490cc4067b7a8a48be24032d2" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37121 } }
I20260808 13:03:49.609236 8675 sys_catalog.cc:564] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:03:49.613076 8685 sys_catalog.cc:455] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c38a240490cc4067b7a8a48be24032d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38a240490cc4067b7a8a48be24032d2" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37121 } } }
I20260808 13:03:49.613054 8686 sys_catalog.cc:455] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c38a240490cc4067b7a8a48be24032d2. Latest consensus state: current_term: 1 leader_uuid: "c38a240490cc4067b7a8a48be24032d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c38a240490cc4067b7a8a48be24032d2" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37121 } } }
I20260808 13:03:49.613514 8685 sys_catalog.cc:458] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2 [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:49.613571 8686 sys_catalog.cc:458] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2 [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:49.615096 8692 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:03:49.620741 8692 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:03:49.630435 8692 catalog_manager.cc:1347] Generated new cluster ID: 34117067fa174f7589d87357f621e8cc
I20260808 13:03:49.630576 8692 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:03:49.665666 8692 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:03:49.666117 8692 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:03:49.678898 8692 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2: Generated new TSK 0
I20260808 13:03:49.679786 8692 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:03:49.694129 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:37121
--builtin_ntp_servers=127.1.124.148:33341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 13:03:50.055078 8711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:50.055362 8711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:50.055522 8711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:50.062876 8711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:50.062978 8711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:50.063151 8711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:03:50.072404 8711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:33341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:37121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:50.073747 8711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:50.075626 8711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:50.089046 8728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:50.089082 8732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:50.089309 8734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:50.090092 8711 server_base.cc:1034] running on GCE node
I20260808 13:03:50.090631 8711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:50.091599 8711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:50.092829 8711 hybrid_clock.cc:648] HybridClock initialized: now 1786194230092771 us; error 58 us; skew 500 ppm
I20260808 13:03:50.093215 8711 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:03:50.095504 8711 webserver.cc:460] Webserver started at http://127.1.124.129:45053/ using document root <none> and password file <none>
I20260808 13:03:50.096261 8711 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:50.096397 8711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:50.096735 8711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:50.099565 8711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "609d3fba289f4ba5a2772ab83d7dd6ad"
format_stamp: "Formatted at 2026-08-08 13:03:50 on dist-test-slave-t3q3"
server_key: "b0432d46fc89cdc0856a1f4153178ad5"
server_key_iv: "04c57dee995c9a1bffbaaf84cd7635b9"
server_key_version: "encryptionkey@0"
I20260808 13:03:50.100449 8711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "609d3fba289f4ba5a2772ab83d7dd6ad"
format_stamp: "Formatted at 2026-08-08 13:03:50 on dist-test-slave-t3q3"
server_key: "b0432d46fc89cdc0856a1f4153178ad5"
server_key_iv: "04c57dee995c9a1bffbaaf84cd7635b9"
server_key_version: "encryptionkey@0"
I20260808 13:03:50.105751 8711 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:03:50.109288 8743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:50.110785 8711 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:03:50.111008 8711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "609d3fba289f4ba5a2772ab83d7dd6ad"
format_stamp: "Formatted at 2026-08-08 13:03:50 on dist-test-slave-t3q3"
server_key: "b0432d46fc89cdc0856a1f4153178ad5"
server_key_iv: "04c57dee995c9a1bffbaaf84cd7635b9"
server_key_version: "encryptionkey@0"
I20260808 13:03:50.111310 8711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:50.117329 8711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:50.118191 8711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:50.118501 8711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:50.119515 8711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:03:50.121040 8711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:50.121167 8711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:50.121306 8711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:50.121409 8711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:50.155403 8711 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:32777
I20260808 13:03:50.155460 8929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:32777 every 8 connection(s)
I20260808 13:03:50.156803 8711 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:03:50.162317 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 8711
I20260808 13:03:50.162554 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:03:50.163259 1522 external_mini_cluster.cc:1428] Setting key 9a69076cd6a3e7eaaf40356b793da0ff
I20260808 13:03:50.165336 8936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:50.159418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:54583 (local address 127.1.124.190:37121)
0808 13:03:50.159933 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:50.159942 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:50.161107 (+ 1165us) server_negotiation.cc:407] Connection header received
0808 13:03:50.162906 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:50.162913 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:50.162983 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:50.163116 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:50.164527 (+ 1411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:50.164534 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:50.164539 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:03:50.164559 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:50.164612 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:03:50.164624 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:50.164758 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:50.164964 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 13:03:50.165033 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":235,"threads_started":1}
I20260808 13:03:50.166168 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:37121
--builtin_ntp_servers=127.1.124.148:33341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 13:03:50.166246 8937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:50.160008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:37121 (local address 127.1.124.129:54583)
0808 13:03:50.160954 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:50.160998 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:50.162591 (+ 1593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:50.163389 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:50.163397 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:50.163488 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:50.164221 (+ 733us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:03:50.164224 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:50.164261 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:03:50.164347 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:50.164733 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:50.164745 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:50.164753 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 13:03:50.165046 (+ 293us) client_negotiation.cc:240] Negotiation successful
0808 13:03:50.165305 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":268,"threads_started":1}
I20260808 13:03:50.167963 8933 heartbeater.cc:347] Connected to a master server at 127.1.124.190:37121
I20260808 13:03:50.168300 8933 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:50.168886 8933 heartbeater.cc:511] Master 127.1.124.190:37121 requested a full tablet report, sending...
I20260808 13:03:50.171134 8612 ts_manager.cc:194] Registered new tserver with Master: 609d3fba289f4ba5a2772ab83d7dd6ad (127.1.124.129:32777)
W20260808 13:03:50.505184 8938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:50.505488 8938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:50.505663 8938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:50.513273 8938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:50.513387 8938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:50.513574 8938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:03:50.523402 8938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:33341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:37121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:50.524859 8938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:50.526721 8938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:50.540666 8947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:50.540709 8949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:50.540748 8955 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:50.545851 8938 server_base.cc:1034] running on GCE node
I20260808 13:03:50.546584 8938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:50.547729 8938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:50.548974 8938 hybrid_clock.cc:648] HybridClock initialized: now 1786194230548920 us; error 64 us; skew 500 ppm
I20260808 13:03:50.549355 8938 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:03:50.551970 8938 webserver.cc:460] Webserver started at http://127.1.124.130:41353/ using document root <none> and password file <none>
I20260808 13:03:50.552744 8938 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:50.552881 8938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:50.553253 8938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:50.556457 8938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "b30e49779e36415f93ba96923bc4f9c3"
format_stamp: "Formatted at 2026-08-08 13:03:50 on dist-test-slave-t3q3"
server_key: "0fbb28259984a5f98e1d91ee7e332243"
server_key_iv: "94c941752c4f19248ab03c69e8324d54"
server_key_version: "encryptionkey@0"
I20260808 13:03:50.557286 8938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "b30e49779e36415f93ba96923bc4f9c3"
format_stamp: "Formatted at 2026-08-08 13:03:50 on dist-test-slave-t3q3"
server_key: "0fbb28259984a5f98e1d91ee7e332243"
server_key_iv: "94c941752c4f19248ab03c69e8324d54"
server_key_version: "encryptionkey@0"
I20260808 13:03:50.562886 8938 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:03:50.566596 8963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:50.568356 8938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:03:50.568549 8938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "b30e49779e36415f93ba96923bc4f9c3"
format_stamp: "Formatted at 2026-08-08 13:03:50 on dist-test-slave-t3q3"
server_key: "0fbb28259984a5f98e1d91ee7e332243"
server_key_iv: "94c941752c4f19248ab03c69e8324d54"
server_key_version: "encryptionkey@0"
I20260808 13:03:50.568826 8938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:50.576485 8938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:50.577458 8938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:50.577741 8938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:50.578896 8938 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:03:50.580715 8938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:50.580824 8938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:50.580924 8938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:50.581005 8938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:50.620728 8938 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:44347
I20260808 13:03:50.620770 9135 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:44347 every 8 connection(s)
I20260808 13:03:50.622267 8938 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:03:50.630280 8936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:50.624846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:42913 (local address 127.1.124.190:37121)
0808 13:03:50.625057 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:50.625062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:50.626438 (+ 1376us) server_negotiation.cc:407] Connection header received
0808 13:03:50.628184 (+ 1746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:50.628189 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:50.628245 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:50.628339 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:50.629614 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:50.629619 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:50.629622 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:03:50.629639 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:50.629673 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:03:50.629685 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:50.629808 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:50.629989 (+ 181us) server_negotiation.cc:299] Negotiation successful
0808 13:03:50.630050 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260808 13:03:50.631156 9142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:50.625375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:37121 (local address 127.1.124.130:42913)
0808 13:03:50.626289 (+ 914us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:50.626325 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:50.627848 (+ 1523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:50.628541 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:50.628548 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:50.628632 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:50.629313 (+ 681us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:03:50.629315 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:50.629359 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:03:50.629437 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:50.629778 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:50.629786 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:50.629791 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 13:03:50.630006 (+ 215us) client_negotiation.cc:240] Negotiation successful
0808 13:03:50.630192 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":288,"threads_started":1}
I20260808 13:03:50.631518 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 8938
I20260808 13:03:50.631709 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:03:50.632287 1522 external_mini_cluster.cc:1428] Setting key 2591020fb3ae8fd3a437bbc454190869
I20260808 13:03:50.632715 9138 heartbeater.cc:347] Connected to a master server at 127.1.124.190:37121
I20260808 13:03:50.632984 9138 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:50.633603 9138 heartbeater.cc:511] Master 127.1.124.190:37121 requested a full tablet report, sending...
I20260808 13:03:50.635074 8612 ts_manager.cc:194] Registered new tserver with Master: b30e49779e36415f93ba96923bc4f9c3 (127.1.124.130:44347)
I20260808 13:03:50.635185 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:37121
--builtin_ntp_servers=127.1.124.148:33341
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 13:03:50.977150 9144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:50.977435 9144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:50.977526 9144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:50.984789 9144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:50.984901 9144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:50.985011 9144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:03:50.994047 9144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:33341
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:37121
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:50.995319 9144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:50.997138 9144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:51.011107 9150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:51.011184 9156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:51.011152 9152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:51.012697 9144 server_base.cc:1034] running on GCE node
I20260808 13:03:51.013356 9144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:51.014374 9144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:51.015616 9144 hybrid_clock.cc:648] HybridClock initialized: now 1786194231015558 us; error 52 us; skew 500 ppm
I20260808 13:03:51.016011 9144 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:03:51.018777 9144 webserver.cc:460] Webserver started at http://127.1.124.131:41277/ using document root <none> and password file <none>
I20260808 13:03:51.019557 9144 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:51.019652 9144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:51.019987 9144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:51.023144 9144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "906433805e694acd9349072a5e878e28"
format_stamp: "Formatted at 2026-08-08 13:03:51 on dist-test-slave-t3q3"
server_key: "ab409276fce089b2a0ac4e9b3d4b7d5a"
server_key_iv: "9fbb51980f0dc576ce6eef645289bedb"
server_key_version: "encryptionkey@0"
I20260808 13:03:51.024147 9144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "906433805e694acd9349072a5e878e28"
format_stamp: "Formatted at 2026-08-08 13:03:51 on dist-test-slave-t3q3"
server_key: "ab409276fce089b2a0ac4e9b3d4b7d5a"
server_key_iv: "9fbb51980f0dc576ce6eef645289bedb"
server_key_version: "encryptionkey@0"
I20260808 13:03:51.029747 9144 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:03:51.033432 9171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:51.034879 9144 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:03:51.035086 9144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "906433805e694acd9349072a5e878e28"
format_stamp: "Formatted at 2026-08-08 13:03:51 on dist-test-slave-t3q3"
server_key: "ab409276fce089b2a0ac4e9b3d4b7d5a"
server_key_iv: "9fbb51980f0dc576ce6eef645289bedb"
server_key_version: "encryptionkey@0"
I20260808 13:03:51.035337 9144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:51.041205 9144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:51.042023 9144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:51.042309 9144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:51.043229 9144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:03:51.044690 9144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:03:51.044818 9144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:51.044962 9144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:03:51.045054 9144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:51.079972 9144 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:37841
I20260808 13:03:51.080029 9354 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:37841 every 8 connection(s)
I20260808 13:03:51.081346 9144 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:03:51.089345 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 9144
I20260808 13:03:51.089627 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:03:51.089983 8936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:51.084052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:35605 (local address 127.1.124.190:37121)
0808 13:03:51.084288 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:51.084293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:51.085886 (+ 1593us) server_negotiation.cc:407] Connection header received
0808 13:03:51.087536 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:51.087541 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:51.087594 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:51.087698 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:51.089235 (+ 1537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:51.089241 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:51.089244 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:03:51.089260 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:51.089293 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:03:51.089305 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:51.089415 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:51.089679 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 13:03:51.089767 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 13:03:51.090189 1522 external_mini_cluster.cc:1428] Setting key 816ab85cd6caa3988a8664b117615770
I20260808 13:03:51.091261 9361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:51.084623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:37121 (local address 127.1.124.131:35605)
0808 13:03:51.085705 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:51.085758 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:51.087296 (+ 1538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:51.087901 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:51.087909 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:51.087997 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:51.088791 (+ 794us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:03:51.088794 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:51.088836 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:03:51.088991 (+ 155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:51.089421 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:51.089433 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:51.089442 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 13:03:51.089760 (+ 318us) client_negotiation.cc:240] Negotiation successful
0808 13:03:51.090029 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":353,"threads_started":1}
I20260808 13:03:51.092805 9356 heartbeater.cc:347] Connected to a master server at 127.1.124.190:37121
I20260808 13:03:51.093164 9356 heartbeater.cc:464] Registering TS with master...
I20260808 13:03:51.093879 9356 heartbeater.cc:511] Master 127.1.124.190:37121 requested a full tablet report, sending...
I20260808 13:03:51.095268 8612 ts_manager.cc:194] Registered new tserver with Master: 906433805e694acd9349072a5e878e28 (127.1.124.131:37841)
I20260808 13:03:51.105883 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:03:51.113255 8936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:51.110482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41570 (local address 127.1.124.190:37121)
0808 13:03:51.110670 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:51.110675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:51.111250 (+ 575us) server_negotiation.cc:407] Connection header received
0808 13:03:51.111463 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:51.111468 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:51.111520 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:51.111635 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:51.112595 (+ 960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:51.112602 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:51.112607 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:03:51.112630 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:51.112661 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:03:51.112673 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:51.112811 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:51.112901 (+ 90us) server_negotiation.cc:299] Negotiation successful
0808 13:03:51.112977 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 13:03:51.117443 8612 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41570:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:03:51.124583 8612 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:03:51.142589 9378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:51.137008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37832 (local address 127.1.124.130:44347)
0808 13:03:51.138484 (+ 1476us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:51.138494 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:51.138522 (+ 28us) server_negotiation.cc:407] Connection header received
0808 13:03:51.138595 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:51.138602 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:51.138921 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:51.139977 (+ 1056us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:51.141698 (+ 1721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:51.141707 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:51.141727 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:03:51.141742 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:51.141785 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:03:51.141799 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:51.141964 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:51.142134 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 13:03:51.142255 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1277,"thread_start_us":1128,"threads_started":1}
I20260808 13:03:51.142752 9377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:51.136883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:44347 (local address 127.0.0.1:37832)
0808 13:03:51.137749 (+ 866us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:51.137798 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:51.137921 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:51.139200 (+ 1279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:51.139211 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:51.139300 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:51.140362 (+ 1062us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:03:51.140368 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:51.140417 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:03:51.140534 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:51.141914 (+ 1380us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:51.141928 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:51.141936 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 13:03:51.142143 (+ 207us) client_negotiation.cc:240] Negotiation successful
0808 13:03:51.142231 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":655,"threads_started":1}
I20260808 13:03:51.143662 9379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:51.137320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:37841 (local address 127.0.0.1:49974)
0808 13:03:51.138828 (+ 1508us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:51.138852 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:51.138974 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:51.140553 (+ 1579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:51.140560 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:51.140586 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:51.141301 (+ 715us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:03:51.141304 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:51.141321 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:03:51.141361 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:51.143075 (+ 1714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:51.143080 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:51.143085 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 13:03:51.143224 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 13:03:51.143272 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1309,"thread_start_us":1132,"threads_started":1}
I20260808 13:03:51.144649 9381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:51.137419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49974 (local address 127.1.124.131:37841)
0808 13:03:51.139612 (+ 2193us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:51.139621 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:51.139654 (+ 33us) server_negotiation.cc:407] Connection header received
0808 13:03:51.139728 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:51.139737 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:51.140359 (+ 622us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:51.141526 (+ 1167us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:51.142554 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:51.142563 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:51.142578 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:03:51.142606 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:51.142661 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:03:51.142683 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:51.144121 (+ 1438us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:51.144206 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 13:03:51.144293 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2018,"thread_start_us":1858,"threads_started":1}
I20260808 13:03:51.145134 9382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:51.139345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:32777 (local address 127.0.0.1:39788)
0808 13:03:51.139891 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:51.139909 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:51.139997 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:51.142294 (+ 2297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:51.142299 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:51.142326 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:51.143066 (+ 740us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:03:51.143070 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:51.143085 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:03:51.143125 (+ 40us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:51.144452 (+ 1327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:51.144456 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:51.144462 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 13:03:51.144584 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 13:03:51.144639 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"thread_start_us":745,"threads_started":1}
I20260808 13:03:51.145839 9383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:51.139895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39788 (local address 127.1.124.129:32777)
0808 13:03:51.141782 (+ 1887us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:51.141791 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:51.141820 (+ 29us) server_negotiation.cc:407] Connection header received
0808 13:03:51.141893 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:51.141900 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:51.142104 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:51.142801 (+ 697us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:51.143948 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:51.143955 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:51.144209 (+ 254us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:03:51.144233 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:51.144294 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:03:51.144327 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:51.145404 (+ 1077us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:51.145470 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 13:03:51.145542 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1738,"thread_start_us":1585,"threads_started":1}
I20260808 13:03:51.146797 9039 tablet_service.cc:1467] Processing CreateTablet for tablet 8c03b8611bf2407a9070bddd9bea6b58 (DEFAULT_TABLE table=test-table [id=244a245682884f24b70b2f7b01bd84db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:51.148176 9039 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c03b8611bf2407a9070bddd9bea6b58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:51.148499 9248 tablet_service.cc:1467] Processing CreateTablet for tablet 8c03b8611bf2407a9070bddd9bea6b58 (DEFAULT_TABLE table=test-table [id=244a245682884f24b70b2f7b01bd84db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:51.149259 8819 tablet_service.cc:1467] Processing CreateTablet for tablet 8c03b8611bf2407a9070bddd9bea6b58 (DEFAULT_TABLE table=test-table [id=244a245682884f24b70b2f7b01bd84db]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:03:51.149899 9248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c03b8611bf2407a9070bddd9bea6b58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:51.150488 8819 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c03b8611bf2407a9070bddd9bea6b58. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:51.166728 9385 tablet_bootstrap.cc:492] T 8c03b8611bf2407a9070bddd9bea6b58 P b30e49779e36415f93ba96923bc4f9c3: Bootstrap starting.
I20260808 13:03:51.168310 9390 tablet_bootstrap.cc:492] T 8c03b8611bf2407a9070bddd9bea6b58 P 609d3fba289f4ba5a2772ab83d7dd6ad: Bootstrap starting.
I20260808 13:03:51.169518 9389 tablet_bootstrap.cc:492] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28: Bootstrap starting.
I20260808 13:03:51.170867 9385 tablet_bootstrap.cc:654] T 8c03b8611bf2407a9070bddd9bea6b58 P b30e49779e36415f93ba96923bc4f9c3: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:51.171900 9385 log.cc:824] T 8c03b8611bf2407a9070bddd9bea6b58 P b30e49779e36415f93ba96923bc4f9c3: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:51.172108 9390 tablet_bootstrap.cc:654] T 8c03b8611bf2407a9070bddd9bea6b58 P 609d3fba289f4ba5a2772ab83d7dd6ad: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:51.173162 9389 tablet_bootstrap.cc:654] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:51.173454 9390 log.cc:824] T 8c03b8611bf2407a9070bddd9bea6b58 P 609d3fba289f4ba5a2772ab83d7dd6ad: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:51.174396 8933 heartbeater.cc:503] Master 127.1.124.190:37121 was elected leader, sending a full tablet report...
I20260808 13:03:51.174993 9385 tablet_bootstrap.cc:492] T 8c03b8611bf2407a9070bddd9bea6b58 P b30e49779e36415f93ba96923bc4f9c3: No bootstrap required, opened a new log
I20260808 13:03:51.175309 9385 ts_tablet_manager.cc:1397] T 8c03b8611bf2407a9070bddd9bea6b58 P b30e49779e36415f93ba96923bc4f9c3: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260808 13:03:51.177065 9390 tablet_bootstrap.cc:492] T 8c03b8611bf2407a9070bddd9bea6b58 P 609d3fba289f4ba5a2772ab83d7dd6ad: No bootstrap required, opened a new log
I20260808 13:03:51.177335 9390 ts_tablet_manager.cc:1397] T 8c03b8611bf2407a9070bddd9bea6b58 P 609d3fba289f4ba5a2772ab83d7dd6ad: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260808 13:03:51.178830 9389 log.cc:824] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:51.182322 9389 tablet_bootstrap.cc:492] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28: No bootstrap required, opened a new log
I20260808 13:03:51.182601 9389 ts_tablet_manager.cc:1397] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.000s
I20260808 13:03:51.183519 9385 raft_consensus.cc:348] T 8c03b8611bf2407a9070bddd9bea6b58 P b30e49779e36415f93ba96923bc4f9c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b30e49779e36415f93ba96923bc4f9c3" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 44347 } } peers { permanent_uuid: "906433805e694acd9349072a5e878e28" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37841 } } peers { permanent_uuid: "609d3fba289f4ba5a2772ab83d7dd6ad" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 32777 } }
I20260808 13:03:51.183897 9385 raft_consensus.cc:374] T 8c03b8611bf2407a9070bddd9bea6b58 P b30e49779e36415f93ba96923bc4f9c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:51.184003 9385 raft_consensus.cc:729] T 8c03b8611bf2407a9070bddd9bea6b58 P b30e49779e36415f93ba96923bc4f9c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b30e49779e36415f93ba96923bc4f9c3, State: Initialized, Role: FOLLOWER
I20260808 13:03:51.184687 9385 consensus_queue.cc:260] T 8c03b8611bf2407a9070bddd9bea6b58 P b30e49779e36415f93ba96923bc4f9c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b30e49779e36415f93ba96923bc4f9c3" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 44347 } } peers { permanent_uuid: "906433805e694acd9349072a5e878e28" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37841 } } peers { permanent_uuid: "609d3fba289f4ba5a2772ab83d7dd6ad" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 32777 } }
I20260808 13:03:51.185781 9138 heartbeater.cc:503] Master 127.1.124.190:37121 was elected leader, sending a full tablet report...
I20260808 13:03:51.186430 9385 ts_tablet_manager.cc:1428] T 8c03b8611bf2407a9070bddd9bea6b58 P b30e49779e36415f93ba96923bc4f9c3: Time spent starting tablet: real 0.011s user 0.007s sys 0.003s
I20260808 13:03:51.189019 9390 raft_consensus.cc:348] T 8c03b8611bf2407a9070bddd9bea6b58 P 609d3fba289f4ba5a2772ab83d7dd6ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b30e49779e36415f93ba96923bc4f9c3" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 44347 } } peers { permanent_uuid: "906433805e694acd9349072a5e878e28" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37841 } } peers { permanent_uuid: "609d3fba289f4ba5a2772ab83d7dd6ad" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 32777 } }
I20260808 13:03:51.189533 9390 raft_consensus.cc:374] T 8c03b8611bf2407a9070bddd9bea6b58 P 609d3fba289f4ba5a2772ab83d7dd6ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:51.189636 9390 raft_consensus.cc:729] T 8c03b8611bf2407a9070bddd9bea6b58 P 609d3fba289f4ba5a2772ab83d7dd6ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 609d3fba289f4ba5a2772ab83d7dd6ad, State: Initialized, Role: FOLLOWER
I20260808 13:03:51.190286 9390 consensus_queue.cc:260] T 8c03b8611bf2407a9070bddd9bea6b58 P 609d3fba289f4ba5a2772ab83d7dd6ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b30e49779e36415f93ba96923bc4f9c3" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 44347 } } peers { permanent_uuid: "906433805e694acd9349072a5e878e28" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37841 } } peers { permanent_uuid: "609d3fba289f4ba5a2772ab83d7dd6ad" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 32777 } }
I20260808 13:03:51.192221 9390 ts_tablet_manager.cc:1428] T 8c03b8611bf2407a9070bddd9bea6b58 P 609d3fba289f4ba5a2772ab83d7dd6ad: Time spent starting tablet: real 0.015s user 0.009s sys 0.006s
I20260808 13:03:51.193039 9389 raft_consensus.cc:348] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b30e49779e36415f93ba96923bc4f9c3" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 44347 } } peers { permanent_uuid: "906433805e694acd9349072a5e878e28" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37841 } } peers { permanent_uuid: "609d3fba289f4ba5a2772ab83d7dd6ad" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 32777 } }
I20260808 13:03:51.193419 9389 raft_consensus.cc:374] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:51.193549 9389 raft_consensus.cc:729] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 906433805e694acd9349072a5e878e28, State: Initialized, Role: FOLLOWER
I20260808 13:03:51.194161 9389 consensus_queue.cc:260] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b30e49779e36415f93ba96923bc4f9c3" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 44347 } } peers { permanent_uuid: "906433805e694acd9349072a5e878e28" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37841 } } peers { permanent_uuid: "609d3fba289f4ba5a2772ab83d7dd6ad" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 32777 } }
I20260808 13:03:51.195595 9356 heartbeater.cc:503] Master 127.1.124.190:37121 was elected leader, sending a full tablet report...
I20260808 13:03:51.196249 9389 ts_tablet_manager.cc:1428] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28: Time spent starting tablet: real 0.014s user 0.016s sys 0.000s
I20260808 13:03:51.319747 9399 raft_consensus.cc:482] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:03:51.320145 9399 raft_consensus.cc:504] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b30e49779e36415f93ba96923bc4f9c3" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 44347 } } peers { permanent_uuid: "906433805e694acd9349072a5e878e28" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37841 } } peers { permanent_uuid: "609d3fba289f4ba5a2772ab83d7dd6ad" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 32777 } }
I20260808 13:03:51.321842 9399 leader_election.cc:288] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b30e49779e36415f93ba96923bc4f9c3 (127.1.124.130:44347), 609d3fba289f4ba5a2772ab83d7dd6ad (127.1.124.129:32777)
I20260808 13:03:51.324750 9361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:51.322342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:44347 (local address 127.1.124.131:46707)
0808 13:03:51.322559 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:51.322587 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:51.322682 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:51.323064 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:51.323069 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:51.323095 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:51.323794 (+ 699us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:03:51.323797 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:51.323812 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:03:51.323856 (+ 44us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:51.324235 (+ 379us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:51.324251 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:51.324258 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 13:03:51.324446 (+ 188us) client_negotiation.cc:240] Negotiation successful
0808 13:03:51.324516 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260808 13:03:51.324928 9378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:51.322455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:46707 (local address 127.1.124.130:44347)
0808 13:03:51.322710 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:51.322716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:51.322740 (+ 24us) server_negotiation.cc:407] Connection header received
0808 13:03:51.322813 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:51.322818 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:51.322890 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:51.323015 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:51.324013 (+ 998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:51.324021 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:51.324029 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:03:51.324054 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:51.324122 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:03:51.324140 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:51.324259 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:51.324465 (+ 206us) server_negotiation.cc:299] Negotiation successful
0808 13:03:51.324532 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 13:03:51.326125 9400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:51.322785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:32777 (local address 127.1.124.131:49681)
0808 13:03:51.323619 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:51.323634 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:51.323772 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:51.324163 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:51.324169 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:51.324212 (+ 43us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:03:51.325215 (+ 1003us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:03:51.325221 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:03:51.325240 (+ 19us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:03:51.325285 (+ 45us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:03:51.325685 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:03:51.325688 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:03:51.325692 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 13:03:51.325791 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 13:03:51.325830 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":741,"thread_start_us":573,"threads_started":1}
I20260808 13:03:51.326216 9383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:51.323055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:49681 (local address 127.1.124.129:32777)
0808 13:03:51.323287 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:51.323295 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:51.323730 (+ 435us) server_negotiation.cc:407] Connection header received
0808 13:03:51.323906 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:51.323913 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:51.323979 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:51.324124 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:51.325443 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:51.325448 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:51.325451 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:03:51.325488 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:51.325548 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:03:51.325570 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:51.325708 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:51.325788 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 13:03:51.325936 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 13:03:51.326228 9066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c03b8611bf2407a9070bddd9bea6b58" candidate_uuid: "906433805e694acd9349072a5e878e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b30e49779e36415f93ba96923bc4f9c3" is_pre_election: true
I20260808 13:03:51.326848 9066 raft_consensus.cc:2454] T 8c03b8611bf2407a9070bddd9bea6b58 P b30e49779e36415f93ba96923bc4f9c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 906433805e694acd9349072a5e878e28 in term 0.
I20260808 13:03:51.327283 8852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c03b8611bf2407a9070bddd9bea6b58" candidate_uuid: "906433805e694acd9349072a5e878e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "609d3fba289f4ba5a2772ab83d7dd6ad" is_pre_election: true
I20260808 13:03:51.327644 9173 leader_election.cc:302] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 906433805e694acd9349072a5e878e28, b30e49779e36415f93ba96923bc4f9c3; no voters:
I20260808 13:03:51.327885 8852 raft_consensus.cc:2454] T 8c03b8611bf2407a9070bddd9bea6b58 P 609d3fba289f4ba5a2772ab83d7dd6ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 906433805e694acd9349072a5e878e28 in term 0.
I20260808 13:03:51.328049 9399 raft_consensus.cc:2781] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:03:51.328286 9399 raft_consensus.cc:482] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:03:51.328411 9399 raft_consensus.cc:3037] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:51.330689 9399 raft_consensus.cc:504] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b30e49779e36415f93ba96923bc4f9c3" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 44347 } } peers { permanent_uuid: "906433805e694acd9349072a5e878e28" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37841 } } peers { permanent_uuid: "609d3fba289f4ba5a2772ab83d7dd6ad" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 32777 } }
I20260808 13:03:51.331276 9399 leader_election.cc:288] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28 [CANDIDATE]: Term 1 election: Requested vote from peers b30e49779e36415f93ba96923bc4f9c3 (127.1.124.130:44347), 609d3fba289f4ba5a2772ab83d7dd6ad (127.1.124.129:32777)
I20260808 13:03:51.331810 9066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c03b8611bf2407a9070bddd9bea6b58" candidate_uuid: "906433805e694acd9349072a5e878e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b30e49779e36415f93ba96923bc4f9c3"
I20260808 13:03:51.331878 8852 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c03b8611bf2407a9070bddd9bea6b58" candidate_uuid: "906433805e694acd9349072a5e878e28" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "609d3fba289f4ba5a2772ab83d7dd6ad"
I20260808 13:03:51.332024 9066 raft_consensus.cc:3037] T 8c03b8611bf2407a9070bddd9bea6b58 P b30e49779e36415f93ba96923bc4f9c3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:51.332116 8852 raft_consensus.cc:3037] T 8c03b8611bf2407a9070bddd9bea6b58 P 609d3fba289f4ba5a2772ab83d7dd6ad [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:03:51.333107 9360 tablet.cc:2369] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:03:51.334448 8852 raft_consensus.cc:2454] T 8c03b8611bf2407a9070bddd9bea6b58 P 609d3fba289f4ba5a2772ab83d7dd6ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 906433805e694acd9349072a5e878e28 in term 1.
I20260808 13:03:51.334860 9173 leader_election.cc:302] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 609d3fba289f4ba5a2772ab83d7dd6ad, 906433805e694acd9349072a5e878e28; no voters:
I20260808 13:03:51.334950 9066 raft_consensus.cc:2454] T 8c03b8611bf2407a9070bddd9bea6b58 P b30e49779e36415f93ba96923bc4f9c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 906433805e694acd9349072a5e878e28 in term 1.
I20260808 13:03:51.335183 9399 raft_consensus.cc:2781] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:51.335714 9399 raft_consensus.cc:686] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28 [term 1 LEADER]: Becoming Leader. State: Replica: 906433805e694acd9349072a5e878e28, State: Running, Role: LEADER
I20260808 13:03:51.336294 9399 consensus_queue.cc:237] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b30e49779e36415f93ba96923bc4f9c3" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 44347 } } peers { permanent_uuid: "906433805e694acd9349072a5e878e28" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37841 } } peers { permanent_uuid: "609d3fba289f4ba5a2772ab83d7dd6ad" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 32777 } }
I20260808 13:03:51.341032 8611 catalog_manager.cc:5495] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28 reported cstate change: term changed from 0 to 1, leader changed from <none> to 906433805e694acd9349072a5e878e28 (127.1.124.131). New cstate: current_term: 1 leader_uuid: "906433805e694acd9349072a5e878e28" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b30e49779e36415f93ba96923bc4f9c3" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 44347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "906433805e694acd9349072a5e878e28" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "609d3fba289f4ba5a2772ab83d7dd6ad" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 32777 } health_report { overall_health: UNKNOWN } } }
W20260808 13:03:51.374289 9141 tablet.cc:2369] T 8c03b8611bf2407a9070bddd9bea6b58 P b30e49779e36415f93ba96923bc4f9c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:03:51.382923 9381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:51.380874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49982 (local address 127.1.124.131:37841)
0808 13:03:51.381047 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:51.381052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:51.381192 (+ 140us) server_negotiation.cc:407] Connection header received
0808 13:03:51.381373 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:51.381377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:51.381456 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:51.381542 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:51.382455 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:51.382459 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:51.382462 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:03:51.382476 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:51.382507 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:03:51.382518 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:51.382617 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:51.382700 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 13:03:51.382744 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 13:03:51.391587 8852 raft_consensus.cc:1261] T 8c03b8611bf2407a9070bddd9bea6b58 P 609d3fba289f4ba5a2772ab83d7dd6ad [term 1 FOLLOWER]: Refusing update from remote peer 906433805e694acd9349072a5e878e28: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:03:51.391587 9066 raft_consensus.cc:1261] T 8c03b8611bf2407a9070bddd9bea6b58 P b30e49779e36415f93ba96923bc4f9c3 [term 1 FOLLOWER]: Refusing update from remote peer 906433805e694acd9349072a5e878e28: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:03:51.392596 9401 consensus_queue.cc:1035] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b30e49779e36415f93ba96923bc4f9c3" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 44347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:03:51.392998 9399 consensus_queue.cc:1035] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28 [LEADER]: Connected to new peer: Peer: permanent_uuid: "609d3fba289f4ba5a2772ab83d7dd6ad" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 32777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:03:51.406895 9408 mvcc.cc:205] Tried to move back new op lower bound from 7316251571769413632 to 7316251571557416960. Current Snapshot: MvccSnapshot[applied={T|T < 7316251571769413632}]
I20260808 13:03:51.425344 8611 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:03:51.425668 8611 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41570:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:03:51.429368 8611 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c38a240490cc4067b7a8a48be24032d2: Sending DeleteTablet for 3 replicas of tablet 8c03b8611bf2407a9070bddd9bea6b58
I20260808 13:03:51.430552 9248 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c03b8611bf2407a9070bddd9bea6b58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:51 UTC) from {username='slave'} at 127.0.0.1:49974
I20260808 13:03:51.430567 8819 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c03b8611bf2407a9070bddd9bea6b58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:51 UTC) from {username='slave'} at 127.0.0.1:39788
I20260808 13:03:51.431522 9039 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c03b8611bf2407a9070bddd9bea6b58 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:03:51 UTC) from {username='slave'} at 127.0.0.1:37832
I20260808 13:03:51.431939 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 8711
I20260808 13:03:51.431953 9421 tablet_replica.cc:330] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28: stopping tablet replica
I20260808 13:03:51.432396 9421 raft_consensus.cc:2229] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:03:51.433076 9421 raft_consensus.cc:2258] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:51.433823 9424 tablet_replica.cc:330] T 8c03b8611bf2407a9070bddd9bea6b58 P 609d3fba289f4ba5a2772ab83d7dd6ad: stopping tablet replica
I20260808 13:03:51.434198 9424 raft_consensus.cc:2229] T 8c03b8611bf2407a9070bddd9bea6b58 P 609d3fba289f4ba5a2772ab83d7dd6ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:03:51.434684 9424 raft_consensus.cc:2258] T 8c03b8611bf2407a9070bddd9bea6b58 P 609d3fba289f4ba5a2772ab83d7dd6ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:51.436870 9424 ts_tablet_manager.cc:1905] T 8c03b8611bf2407a9070bddd9bea6b58 P 609d3fba289f4ba5a2772ab83d7dd6ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:51.440711 9421 ts_tablet_manager.cc:1905] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:51.444016 9424 ts_tablet_manager.cc:1918] T 8c03b8611bf2407a9070bddd9bea6b58 P 609d3fba289f4ba5a2772ab83d7dd6ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:03:51.444221 9424 log.cc:1196] T 8c03b8611bf2407a9070bddd9bea6b58 P 609d3fba289f4ba5a2772ab83d7dd6ad: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/8c03b8611bf2407a9070bddd9bea6b58
I20260808 13:03:51.444783 9424 ts_tablet_manager.cc:1939] T 8c03b8611bf2407a9070bddd9bea6b58 P 609d3fba289f4ba5a2772ab83d7dd6ad: Deleting consensus metadata
I20260808 13:03:51.446772 8587 catalog_manager.cc:4841] TS 609d3fba289f4ba5a2772ab83d7dd6ad (127.1.124.129:32777): tablet 8c03b8611bf2407a9070bddd9bea6b58 (table test-table [id=244a245682884f24b70b2f7b01bd84db]) successfully deleted
I20260808 13:03:51.447239 9427 tablet_replica.cc:330] T 8c03b8611bf2407a9070bddd9bea6b58 P b30e49779e36415f93ba96923bc4f9c3: stopping tablet replica
I20260808 13:03:51.447633 9427 raft_consensus.cc:2229] T 8c03b8611bf2407a9070bddd9bea6b58 P b30e49779e36415f93ba96923bc4f9c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:03:51.448133 9427 raft_consensus.cc:2258] T 8c03b8611bf2407a9070bddd9bea6b58 P b30e49779e36415f93ba96923bc4f9c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:03:51.448567 9383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:51.434000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39804 (local address 127.1.124.129:32777)
0808 13:03:51.434250 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:51.434259 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:51.444042 (+ 9783us) server_negotiation.cc:407] Connection header received
0808 13:03:51.444683 (+ 641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:51.444691 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:51.444839 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:51.444931 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:51.446596 (+ 1665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:51.446603 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:51.446608 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:03:51.446626 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:51.446666 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:03:51.446685 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:51.446815 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:51.448223 (+ 1408us) server_negotiation.cc:299] Negotiation successful
0808 13:03:51.448292 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 13:03:51.449729 8923 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:51.450544 9427 ts_tablet_manager.cc:1905] T 8c03b8611bf2407a9070bddd9bea6b58 P b30e49779e36415f93ba96923bc4f9c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:03:51.453289 9421 ts_tablet_manager.cc:1918] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:03:51.453528 9421 log.cc:1196] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/8c03b8611bf2407a9070bddd9bea6b58
I20260808 13:03:51.454128 9421 ts_tablet_manager.cc:1939] T 8c03b8611bf2407a9070bddd9bea6b58 P 906433805e694acd9349072a5e878e28: Deleting consensus metadata
I20260808 13:03:51.456004 8589 catalog_manager.cc:4841] TS 906433805e694acd9349072a5e878e28 (127.1.124.131:37841): tablet 8c03b8611bf2407a9070bddd9bea6b58 (table test-table [id=244a245682884f24b70b2f7b01bd84db]) successfully deleted
I20260808 13:03:51.459095 9427 ts_tablet_manager.cc:1918] T 8c03b8611bf2407a9070bddd9bea6b58 P b30e49779e36415f93ba96923bc4f9c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:03:51.459287 9427 log.cc:1196] T 8c03b8611bf2407a9070bddd9bea6b58 P b30e49779e36415f93ba96923bc4f9c3: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/8c03b8611bf2407a9070bddd9bea6b58
I20260808 13:03:51.459875 9427 ts_tablet_manager.cc:1939] T 8c03b8611bf2407a9070bddd9bea6b58 P b30e49779e36415f93ba96923bc4f9c3: Deleting consensus metadata
I20260808 13:03:51.461854 8587 catalog_manager.cc:4841] TS b30e49779e36415f93ba96923bc4f9c3 (127.1.124.130:44347): tablet 8c03b8611bf2407a9070bddd9bea6b58 (table test-table [id=244a245682884f24b70b2f7b01bd84db]) successfully deleted
I20260808 13:03:51.541430 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 8711
I20260808 13:03:51.565773 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 8938
I20260808 13:03:51.569161 9378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:51.566696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37840 (local address 127.1.124.130:44347)
0808 13:03:51.566891 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:51.566898 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:51.566921 (+ 23us) server_negotiation.cc:407] Connection header received
0808 13:03:51.567048 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:51.567054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:51.567177 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:51.567302 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:51.568336 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:51.568342 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:51.568346 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:03:51.568366 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:51.568418 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:03:51.568437 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:51.568572 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:51.568783 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 13:03:51.568846 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 13:03:51.569871 9129 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:51.673521 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 8938
I20260808 13:03:51.698256 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 9144
I20260808 13:03:51.701545 9381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:51.699156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49994 (local address 127.1.124.131:37841)
0808 13:03:51.699335 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:51.699340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:51.699358 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:03:51.699577 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:51.699581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:51.699631 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:51.699727 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:51.700760 (+ 1033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:51.700765 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:51.700768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:03:51.700783 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:51.700812 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:03:51.700823 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:51.700932 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:51.701065 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 13:03:51.701110 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 13:03:51.702419 9348 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:51.822119 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 9144
I20260808 13:03:51.848204 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 8555
I20260808 13:03:51.849397 8668 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:51.957849 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 8555
2026-08-08T13:03:51Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2894 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 13:03:52.008491 1522 mini_oidc.cc:152] Starting JWKS server
I20260808 13:03:52.011548 1522 webserver.cc:460] Webserver started at https://127.0.0.1:43229/ using document root <none> and password file <none>
I20260808 13:03:52.011869 1522 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 13:03:52.012596 1522 webserver.cc:460] Webserver started at http://127.0.0.1:38445/ using document root <none> and password file <none>
2026-08-08T13:03:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:03:52Z Disabled control of system clock
I20260808 13:03:52.048593 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:33773
--webserver_interface=127.1.124.190
--webserver_port=0
--builtin_ntp_servers=127.1.124.148:46701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:33773
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43229/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194185613124-1522-0/testchainca.cert with env {}
W20260808 13:03:52.391310 9447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:52.391687 9447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:52.391868 9447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:52.399358 9447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:03:52.399529 9447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:52.399627 9447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:52.399752 9447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:03:52.399834 9447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 13:03:52.399933 9447 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 13:03:52.400027 9447 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:43229/jwks.json
I20260808 13:03:52.410796 9447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:46701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:33773
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:33773
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:43229/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194185613124-1522-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:52.412309 9447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:52.413707 9447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:52.425798 9454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:52.425769 9459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:52.425717 9457 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:52.426666 9447 server_base.cc:1034] running on GCE node
I20260808 13:03:52.427383 9447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:52.428714 9447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:52.429915 9447 hybrid_clock.cc:648] HybridClock initialized: now 1786194232429879 us; error 26 us; skew 500 ppm
I20260808 13:03:52.430217 9447 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:03:52.432477 9447 webserver.cc:460] Webserver started at http://127.1.124.190:38113/ using document root <none> and password file <none>
I20260808 13:03:52.433120 9447 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:52.433211 9447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:52.433455 9447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:52.436398 9447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "3f4366a0dfca4668b62f55df0be52e27"
format_stamp: "Formatted at 2026-08-08 13:03:52 on dist-test-slave-t3q3"
server_key: "44f476878c5da28fdf469284ff045380"
server_key_iv: "2b10125f13153c52d13cf4dea89a621a"
server_key_version: "encryptionkey@0"
I20260808 13:03:52.437232 9447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "3f4366a0dfca4668b62f55df0be52e27"
format_stamp: "Formatted at 2026-08-08 13:03:52 on dist-test-slave-t3q3"
server_key: "44f476878c5da28fdf469284ff045380"
server_key_iv: "2b10125f13153c52d13cf4dea89a621a"
server_key_version: "encryptionkey@0"
I20260808 13:03:52.442170 9447 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:03:52.445401 9465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:52.446631 9447 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:03:52.446817 9447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "3f4366a0dfca4668b62f55df0be52e27"
format_stamp: "Formatted at 2026-08-08 13:03:52 on dist-test-slave-t3q3"
server_key: "44f476878c5da28fdf469284ff045380"
server_key_iv: "2b10125f13153c52d13cf4dea89a621a"
server_key_version: "encryptionkey@0"
I20260808 13:03:52.447016 9447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:52.483243 9447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:52.484268 9447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:52.484612 9447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:52.504341 9447 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:33773
I20260808 13:03:52.504362 9546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:33773 every 8 connection(s)
I20260808 13:03:52.505796 9447 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:03:52.509778 9551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:52.512439 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 9447
I20260808 13:03:52.512650 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:03:52.513490 1522 external_mini_cluster.cc:1428] Setting key 6ede5cada67788a5f56cb8aed52e79aa
I20260808 13:03:52.522733 9551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27: Bootstrap starting.
I20260808 13:03:52.526619 9551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:52.527875 9551 log.cc:824] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:52.529044 9561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:52.515701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53430 (local address 127.1.124.190:33773)
0808 13:03:52.516939 (+ 1238us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:52.516954 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:52.517022 (+ 68us) server_negotiation.cc:407] Connection header received
0808 13:03:52.518853 (+ 1831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:52.518877 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:52.519412 (+ 535us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:52.519767 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:52.520492 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:52.521953 (+ 1461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:52.523935 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:52.524704 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:52.524840 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:52.526113 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:52.526145 (+ 32us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:52.526163 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:03:52.526749 (+ 586us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:52.526846 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:03:52.526867 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:52.526958 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:52.527262 (+ 304us) server_negotiation.cc:299] Negotiation successful
0808 13:03:52.527640 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697,"thread_start_us":456,"threads_started":1}
I20260808 13:03:52.530921 9551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27: No bootstrap required, opened a new log
I20260808 13:03:52.533135 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:03:52.539608 9551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f4366a0dfca4668b62f55df0be52e27" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 33773 } }
I20260808 13:03:52.539951 9551 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:52.540100 9551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f4366a0dfca4668b62f55df0be52e27, State: Initialized, Role: FOLLOWER
I20260808 13:03:52.540690 9551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f4366a0dfca4668b62f55df0be52e27" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 33773 } }
I20260808 13:03:52.540928 9551 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:03:52.541083 9551 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:03:52.541285 9551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:52.543469 9551 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f4366a0dfca4668b62f55df0be52e27" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 33773 } }
I20260808 13:03:52.543977 9551 leader_election.cc:302] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f4366a0dfca4668b62f55df0be52e27; no voters:
I20260808 13:03:52.544713 9551 leader_election.cc:288] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:03:52.544836 9569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:52.545548 9569 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27 [term 1 LEADER]: Becoming Leader. State: Replica: 3f4366a0dfca4668b62f55df0be52e27, State: Running, Role: LEADER
I20260808 13:03:52.546044 9569 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f4366a0dfca4668b62f55df0be52e27" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 33773 } }
I20260808 13:03:52.546828 9551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:03:52.548975 9574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f4366a0dfca4668b62f55df0be52e27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f4366a0dfca4668b62f55df0be52e27" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 33773 } } }
I20260808 13:03:52.549489 9574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27 [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:52.550366 9576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f4366a0dfca4668b62f55df0be52e27. Latest consensus state: current_term: 1 leader_uuid: "3f4366a0dfca4668b62f55df0be52e27" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f4366a0dfca4668b62f55df0be52e27" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 33773 } } }
I20260808 13:03:52.550673 9576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27 [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:52.552865 9589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:03:52.556818 9589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:03:52.565438 9589 catalog_manager.cc:1347] Generated new cluster ID: 2388e160170a433aa5a8f8ac56f908ac
I20260808 13:03:52.565555 9589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:03:52.620592 9589 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:03:52.622548 9589 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:03:52.634966 9589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f4366a0dfca4668b62f55df0be52e27: Generated new TSK 0
I20260808 13:03:52.635891 9589 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 13:03:52.680563 9613 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 13:03:52.680811 9613 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 13:03:52.683100 9561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:52.658969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53444 (local address 127.1.124.190:33773)
0808 13:03:52.659182 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:52.659189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:52.659696 (+ 507us) server_negotiation.cc:407] Connection header received
0808 13:03:52.660091 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:52.660097 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:52.660203 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:52.660370 (+ 167us) server_negotiation.cc:226] Negotiated authn=JWT
0808 13:03:52.660905 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:52.662117 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:52.663527 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:52.664266 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:52.664404 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:52.664968 (+ 564us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 13:03:52.682363 (+ 17395us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43229/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 13:03:52.682795 (+ 432us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53444: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43229/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":73}
W20260808 13:03:52.683233 9561 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53444: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43229/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 13:03:52.684835 9611 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:03:52.658789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:33773 (local address 127.0.0.1:53444)
0808 13:03:52.659584 (+ 795us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:52.659601 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:52.659764 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:52.660402 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:52.660406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:52.660416 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0808 13:03:52.660751 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:52.660760 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:52.662254 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:52.662258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:52.663361 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:52.663373 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:52.664429 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:52.664433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:52.664576 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:52.664651 (+ 75us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 13:03:52.682723 (+ 18072us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43229/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 13:03:52.683019 (+ 296us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.124.190:33773: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43229/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":427,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.1.124.190:33773: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:43229/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 13:03:52.687209 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 9447
I20260808 13:03:52.688520 9541 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:52.793093 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 9447
2026-08-08T13:03:52Z chronyd exiting
I20260808 13:03:52.853984 1522 test_util.cc:175] -----------------------------------------------
I20260808 13:03:52.854156 1522 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786194185613124-1522-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (852 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 13:03:52.857996 1522 mini_oidc.cc:152] Starting JWKS server
I20260808 13:03:52.859917 1522 webserver.cc:460] Webserver started at https://127.0.0.1:42265/ using document root <none> and password file <none>
I20260808 13:03:52.860221 1522 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 13:03:52.860918 1522 webserver.cc:460] Webserver started at http://127.0.0.1:34647/ using document root <none> and password file <none>
2026-08-08T13:03:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:03:52Z Disabled control of system clock
I20260808 13:03:52.896955 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:40613
--webserver_interface=127.1.124.190
--webserver_port=0
--builtin_ntp_servers=127.1.124.148:40925
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:40613
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42265/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194185613124-1522-0/testchainca.cert with env {}
W20260808 13:03:53.240715 9622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:53.241109 9622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:53.241292 9622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:53.249396 9622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:03:53.249552 9622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:53.249624 9622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:53.249711 9622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:03:53.249804 9622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 13:03:53.249897 9622 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 13:03:53.249959 9622 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42265/jwks.json
I20260808 13:03:53.260332 9622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:40925
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:40613
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:40613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42265/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194185613124-1522-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:53.261906 9622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:53.263377 9622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:53.275415 9629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:53.275471 9632 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:53.275588 9636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:53.276893 9622 server_base.cc:1034] running on GCE node
I20260808 13:03:53.277727 9622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:53.279207 9622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:53.280484 9622 hybrid_clock.cc:648] HybridClock initialized: now 1786194233280441 us; error 56 us; skew 500 ppm
I20260808 13:03:53.280871 9622 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:03:53.283349 9622 webserver.cc:460] Webserver started at http://127.1.124.190:46269/ using document root <none> and password file <none>
I20260808 13:03:53.284144 9622 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:53.284286 9622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:53.284621 9622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:53.288012 9622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "2c31a2c26b0043f2b140a9a132d49b8e"
format_stamp: "Formatted at 2026-08-08 13:03:53 on dist-test-slave-t3q3"
server_key: "ebbb60d3a1e6216d67fd6e7d7b14d651"
server_key_iv: "3226c3c05d61908c33fe26dcaf6a4497"
server_key_version: "encryptionkey@0"
I20260808 13:03:53.288884 9622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "2c31a2c26b0043f2b140a9a132d49b8e"
format_stamp: "Formatted at 2026-08-08 13:03:53 on dist-test-slave-t3q3"
server_key: "ebbb60d3a1e6216d67fd6e7d7b14d651"
server_key_iv: "3226c3c05d61908c33fe26dcaf6a4497"
server_key_version: "encryptionkey@0"
I20260808 13:03:53.294070 9622 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:03:53.297848 9644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:53.299284 9622 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:03:53.299492 9622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "2c31a2c26b0043f2b140a9a132d49b8e"
format_stamp: "Formatted at 2026-08-08 13:03:53 on dist-test-slave-t3q3"
server_key: "ebbb60d3a1e6216d67fd6e7d7b14d651"
server_key_iv: "3226c3c05d61908c33fe26dcaf6a4497"
server_key_version: "encryptionkey@0"
I20260808 13:03:53.299741 9622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:53.329875 9622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:53.330801 9622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:53.331139 9622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:53.349839 9622 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:40613
I20260808 13:03:53.349848 9735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:40613 every 8 connection(s)
I20260808 13:03:53.351298 9622 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:03:53.355118 9736 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:53.361176 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 9622
I20260808 13:03:53.361400 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:03:53.362094 1522 external_mini_cluster.cc:1428] Setting key c1914af98bcc0b474dd74457513efc7b
I20260808 13:03:53.366582 9736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e: Bootstrap starting.
I20260808 13:03:53.370332 9736 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:53.371685 9736 log.cc:824] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:53.375082 9736 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e: No bootstrap required, opened a new log
I20260808 13:03:53.376348 9739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:53.363886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48766 (local address 127.1.124.190:40613)
0808 13:03:53.364995 (+ 1109us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:53.365006 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:53.365051 (+ 45us) server_negotiation.cc:407] Connection header received
0808 13:03:53.366755 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:53.366786 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:53.367267 (+ 481us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:53.367585 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:53.368331 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:53.369680 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:53.371637 (+ 1957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:53.372717 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:53.372925 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:53.374066 (+ 1141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:53.374076 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:53.374090 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:03:53.374627 (+ 537us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:53.374703 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:03:53.374723 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:53.374810 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:53.375047 (+ 237us) server_negotiation.cc:299] Negotiation successful
0808 13:03:53.375351 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":664,"thread_start_us":445,"threads_started":1}
I20260808 13:03:53.380566 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:03:53.384116 9736 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c31a2c26b0043f2b140a9a132d49b8e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40613 } }
I20260808 13:03:53.384513 9736 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:53.384634 9736 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c31a2c26b0043f2b140a9a132d49b8e, State: Initialized, Role: FOLLOWER
I20260808 13:03:53.385253 9736 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c31a2c26b0043f2b140a9a132d49b8e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40613 } }
I20260808 13:03:53.385584 9736 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:03:53.385748 9736 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:03:53.385953 9736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:53.388217 9736 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c31a2c26b0043f2b140a9a132d49b8e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40613 } }
I20260808 13:03:53.388664 9736 leader_election.cc:302] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c31a2c26b0043f2b140a9a132d49b8e; no voters:
I20260808 13:03:53.389176 9736 leader_election.cc:288] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:03:53.389348 9742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:53.390172 9742 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e [term 1 LEADER]: Becoming Leader. State: Replica: 2c31a2c26b0043f2b140a9a132d49b8e, State: Running, Role: LEADER
I20260808 13:03:53.390693 9742 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c31a2c26b0043f2b140a9a132d49b8e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40613 } }
I20260808 13:03:53.391386 9736 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:03:53.393373 9744 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c31a2c26b0043f2b140a9a132d49b8e. Latest consensus state: current_term: 1 leader_uuid: "2c31a2c26b0043f2b140a9a132d49b8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c31a2c26b0043f2b140a9a132d49b8e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40613 } } }
I20260808 13:03:53.393764 9744 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:53.394421 9743 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c31a2c26b0043f2b140a9a132d49b8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c31a2c26b0043f2b140a9a132d49b8e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40613 } } }
I20260808 13:03:53.394729 9747 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:03:53.394799 9743 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:53.401726 9747 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:03:53.411376 9747 catalog_manager.cc:1347] Generated new cluster ID: 771e45b2c8ef4a38b043fcc7cf98983e
I20260808 13:03:53.411511 9747 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:03:53.420524 9747 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:03:53.421785 9747 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:03:53.429172 9747 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c31a2c26b0043f2b140a9a132d49b8e: Generated new TSK 0
I20260808 13:03:53.429980 9747 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 13:03:53.464402 9785 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 13:03:53.464546 9785 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 13:03:53.465867 9739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:53.445704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48776 (local address 127.1.124.190:40613)
0808 13:03:53.445890 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:53.445896 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:53.446251 (+ 355us) server_negotiation.cc:407] Connection header received
0808 13:03:53.446450 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:53.446456 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:53.446543 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:53.446709 (+ 166us) server_negotiation.cc:226] Negotiated authn=JWT
0808 13:03:53.447154 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:53.448290 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:53.449676 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:53.450381 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:53.450518 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:53.451017 (+ 499us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 13:03:53.465104 (+ 14087us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42265/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 13:03:53.465575 (+ 471us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48776: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42265/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":69}
W20260808 13:03:53.465999 9739 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48776: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42265/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 13:03:53.466068 9783 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:03:53.445521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:40613 (local address 127.0.0.1:48776)
0808 13:03:53.446143 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:53.446159 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:53.446275 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:53.446733 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:53.446737 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:53.446746 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 13:03:53.446986 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:53.446994 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:53.448428 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:53.448432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:53.449522 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:53.449533 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:53.450564 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:53.450569 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:53.450718 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:53.450751 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 13:03:53.465407 (+ 14656us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42265/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 13:03:53.465703 (+ 296us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.124.190:40613: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42265/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":345,"threads_started":1}
I20260808 13:03:53.467458 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 9622
I20260808 13:03:53.468868 9731 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:53.560492 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 9622
2026-08-08T13:03:53Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (763 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 13:03:53.621421 1522 mini_oidc.cc:152] Starting JWKS server
I20260808 13:03:53.623443 1522 webserver.cc:460] Webserver started at https://127.0.0.1:46793/ using document root <none> and password file <none>
I20260808 13:03:53.623723 1522 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 13:03:53.624442 1522 webserver.cc:460] Webserver started at http://127.0.0.1:45983/ using document root <none> and password file <none>
2026-08-08T13:03:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:03:53Z Disabled control of system clock
I20260808 13:03:53.659205 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:44985
--webserver_interface=127.1.124.190
--webserver_port=0
--builtin_ntp_servers=127.1.124.148:41043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:44985
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46793/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 13:03:54.006944 9794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:54.007364 9794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:54.007539 9794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:54.015256 9794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:03:54.015439 9794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:54.015539 9794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:54.015648 9794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:03:54.015738 9794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 13:03:54.015831 9794 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 13:03:54.015911 9794 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46793/jwks.json
I20260808 13:03:54.025595 9794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:41043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:44985
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:44985
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46793/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:54.027058 9794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:54.028990 9794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:54.041152 9808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:54.041109 9804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:54.041186 9802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:54.041982 9794 server_base.cc:1034] running on GCE node
I20260808 13:03:54.042811 9794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:54.044296 9794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:54.045531 9794 hybrid_clock.cc:648] HybridClock initialized: now 1786194234045476 us; error 65 us; skew 500 ppm
I20260808 13:03:54.045887 9794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:03:54.051611 9794 webserver.cc:460] Webserver started at http://127.1.124.190:43569/ using document root <none> and password file <none>
I20260808 13:03:54.052430 9794 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:54.052567 9794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:54.052915 9794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:54.055939 9794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "ec94d91fbfe54aa297e12618955869a8"
format_stamp: "Formatted at 2026-08-08 13:03:54 on dist-test-slave-t3q3"
server_key: "bc9ce3ca7c0cf8055af3cfc417c9c27f"
server_key_iv: "6eb0319bf513559fdcc1b5bca4491e5b"
server_key_version: "encryptionkey@0"
I20260808 13:03:54.056852 9794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "ec94d91fbfe54aa297e12618955869a8"
format_stamp: "Formatted at 2026-08-08 13:03:54 on dist-test-slave-t3q3"
server_key: "bc9ce3ca7c0cf8055af3cfc417c9c27f"
server_key_iv: "6eb0319bf513559fdcc1b5bca4491e5b"
server_key_version: "encryptionkey@0"
I20260808 13:03:54.062130 9794 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260808 13:03:54.065738 9823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:54.067077 9794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:03:54.067276 9794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "ec94d91fbfe54aa297e12618955869a8"
format_stamp: "Formatted at 2026-08-08 13:03:54 on dist-test-slave-t3q3"
server_key: "bc9ce3ca7c0cf8055af3cfc417c9c27f"
server_key_iv: "6eb0319bf513559fdcc1b5bca4491e5b"
server_key_version: "encryptionkey@0"
I20260808 13:03:54.067502 9794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:54.088665 9794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:54.089512 9794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:54.089843 9794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:54.109078 9794 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:44985
I20260808 13:03:54.109097 9916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:44985 every 8 connection(s)
I20260808 13:03:54.110610 9794 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:03:54.111840 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 9794
I20260808 13:03:54.112047 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:03:54.112939 1522 external_mini_cluster.cc:1428] Setting key 96b6c9e05626d22f70d9e5ee3de3e855
I20260808 13:03:54.115481 9917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:54.126662 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:54.114897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46170 (local address 127.1.124.190:44985)
0808 13:03:54.115900 (+ 1003us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:54.115911 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:54.115951 (+ 40us) server_negotiation.cc:407] Connection header received
0808 13:03:54.117564 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:54.117585 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:54.118027 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:54.118366 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:54.118991 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:54.120441 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:54.122342 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:54.122989 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:54.123119 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:54.124355 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:54.124366 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:54.124379 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:03:54.125016 (+ 637us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:54.125090 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:03:54.125111 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:54.125199 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:54.125451 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 13:03:54.125800 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":594,"thread_start_us":393,"threads_started":1}
I20260808 13:03:54.127914 9917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8: Bootstrap starting.
I20260808 13:03:54.130483 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:03:54.131283 9917 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:54.132323 9917 log.cc:824] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:54.135258 9917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8: No bootstrap required, opened a new log
I20260808 13:03:54.143158 9917 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec94d91fbfe54aa297e12618955869a8" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44985 } }
I20260808 13:03:54.143625 9917 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:54.143749 9917 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec94d91fbfe54aa297e12618955869a8, State: Initialized, Role: FOLLOWER
I20260808 13:03:54.144373 9917 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec94d91fbfe54aa297e12618955869a8" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44985 } }
I20260808 13:03:54.144618 9917 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:03:54.144768 9917 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:03:54.144959 9917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:54.147147 9917 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec94d91fbfe54aa297e12618955869a8" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44985 } }
I20260808 13:03:54.147724 9917 leader_election.cc:302] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec94d91fbfe54aa297e12618955869a8; no voters:
I20260808 13:03:54.148437 9917 leader_election.cc:288] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:03:54.148602 9924 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:54.150151 9924 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8 [term 1 LEADER]: Becoming Leader. State: Replica: ec94d91fbfe54aa297e12618955869a8, State: Running, Role: LEADER
I20260808 13:03:54.150564 9917 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:03:54.150707 9924 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec94d91fbfe54aa297e12618955869a8" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44985 } }
I20260808 13:03:54.153724 9934 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec94d91fbfe54aa297e12618955869a8. Latest consensus state: current_term: 1 leader_uuid: "ec94d91fbfe54aa297e12618955869a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec94d91fbfe54aa297e12618955869a8" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44985 } } }
I20260808 13:03:54.154112 9934 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8 [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:54.153825 9933 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec94d91fbfe54aa297e12618955869a8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec94d91fbfe54aa297e12618955869a8" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44985 } } }
I20260808 13:03:54.154241 9933 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8 [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:54.155920 9950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:03:54.161038 9950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:03:54.170871 9950 catalog_manager.cc:1347] Generated new cluster ID: 1b88eb070a5c49ac9d4e319256af1a92
I20260808 13:03:54.171077 9950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:03:54.188923 9950 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:03:54.190495 9950 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:03:54.200594 9950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec94d91fbfe54aa297e12618955869a8: Generated new TSK 0
I20260808 13:03:54.201948 9950 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:03:54.239248 9921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:54.230154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46172 (local address 127.1.124.190:44985)
0808 13:03:54.230360 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:54.230366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:54.230767 (+ 401us) server_negotiation.cc:407] Connection header received
0808 13:03:54.230990 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:54.230997 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:54.231104 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:54.231280 (+ 176us) server_negotiation.cc:226] Negotiated authn=JWT
0808 13:03:54.231886 (+ 606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:54.233132 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:54.235298 (+ 2166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:54.235885 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:54.235989 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:54.236664 (+ 675us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 13:03:54.238564 (+ 1900us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46793/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 13:03:54.238918 (+ 354us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46172: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46793/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":76}
W20260808 13:03:54.239490 9970 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:03:54.230011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:44985 (local address 127.0.0.1:46172)
0808 13:03:54.230661 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:54.230677 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:54.230803 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:54.231344 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:54.231349 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:54.231362 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0808 13:03:54.231698 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:54.231713 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:54.233356 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:54.233363 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:54.235128 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:54.235146 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:54.236127 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:54.236135 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:54.236337 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:54.236388 (+ 51us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 13:03:54.238772 (+ 2384us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46793/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 13:03:54.239049 (+ 277us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.124.190:44985: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46793/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":349,"threads_started":1}
W20260808 13:03:54.240331 9921 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46172: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46793/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
I20260808 13:03:54.240932 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 9794
I20260808 13:03:54.242158 9909 generic_service.cc:196] Checking for leaks (request via RPC)
E20260808 13:03:54.257068 9973 webserver.cc:533] Webserver: sslize failed:
E20260808 13:03:54.257220 9973 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 13:03:54.339522 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 9794
2026-08-08T13:03:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (783 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T13:03:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:03:54Z Disabled control of system clock
I20260808 13:03:54.440203 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:33951
--webserver_interface=127.1.124.190
--webserver_port=0
--builtin_ntp_servers=127.1.124.148:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:33951
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194185613124-1522-0/testchainca.cert with env {}
W20260808 13:03:54.778776 9978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:03:54.779096 9978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:03:54.779187 9978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:03:54.786492 9978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:03:54.786625 9978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:03:54.786665 9978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:03:54.786721 9978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:03:54.786768 9978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 13:03:54.786826 9978 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 13:03:54.786868 9978 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 13:03:54.796928 9978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:33951
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:33951
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194185613124-1522-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:03:54.798254 9978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:03:54.799542 9978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:03:54.811568 9984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:54.811604 9986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:03:54.811728 9983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:03:54.812430 9978 server_base.cc:1034] running on GCE node
I20260808 13:03:54.813259 9978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:03:54.814672 9978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:03:54.815882 9978 hybrid_clock.cc:648] HybridClock initialized: now 1786194234815837 us; error 34 us; skew 500 ppm
I20260808 13:03:54.816279 9978 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:03:54.818651 9978 webserver.cc:460] Webserver started at http://127.1.124.190:36609/ using document root <none> and password file <none>
I20260808 13:03:54.819337 9978 fs_manager.cc:359] Metadata directory not provided
I20260808 13:03:54.819430 9978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:03:54.819710 9978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:03:54.822796 9978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "92b177da4ed84e2ea3dcbcb3962b500e"
format_stamp: "Formatted at 2026-08-08 13:03:54 on dist-test-slave-t3q3"
server_key: "0bc5f365b2b41e86f2d797af008718f1"
server_key_iv: "d515f819af4a3fb289c0f1a1f1cf97ca"
server_key_version: "encryptionkey@0"
I20260808 13:03:54.823589 9978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "92b177da4ed84e2ea3dcbcb3962b500e"
format_stamp: "Formatted at 2026-08-08 13:03:54 on dist-test-slave-t3q3"
server_key: "0bc5f365b2b41e86f2d797af008718f1"
server_key_iv: "d515f819af4a3fb289c0f1a1f1cf97ca"
server_key_version: "encryptionkey@0"
I20260808 13:03:54.828887 9978 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260808 13:03:54.832489 9996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:03:54.833902 9978 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 13:03:54.834051 9978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "92b177da4ed84e2ea3dcbcb3962b500e"
format_stamp: "Formatted at 2026-08-08 13:03:54 on dist-test-slave-t3q3"
server_key: "0bc5f365b2b41e86f2d797af008718f1"
server_key_iv: "d515f819af4a3fb289c0f1a1f1cf97ca"
server_key_version: "encryptionkey@0"
I20260808 13:03:54.834249 9978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:03:54.862962 9978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:03:54.863795 9978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:03:54.864049 9978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:03:54.882800 9978 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:33951
I20260808 13:03:54.882826 10086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:33951 every 8 connection(s)
I20260808 13:03:54.884164 9978 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:03:54.887838 10089 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:03:54.892669 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 9978
I20260808 13:03:54.892902 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:03:54.893729 1522 external_mini_cluster.cc:1428] Setting key 21efd94f989e34acd8fdbd852aad32db
I20260808 13:03:54.899271 10089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e: Bootstrap starting.
I20260808 13:03:54.902952 10089 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e: Neither blocks nor log segments found. Creating new log.
I20260808 13:03:54.904137 10089 log.cc:824] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e: Log is configured to *not* fsync() on all Append() calls
I20260808 13:03:54.907399 10089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e: No bootstrap required, opened a new log
I20260808 13:03:54.907686 10093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:54.895697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44884 (local address 127.1.124.190:33951)
0808 13:03:54.896737 (+ 1040us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:54.896747 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:54.896793 (+ 46us) server_negotiation.cc:407] Connection header received
0808 13:03:54.898476 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:54.898499 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:54.898965 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:54.899291 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:03:54.899947 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:54.901390 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:54.903280 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:54.903987 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:54.904208 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:54.905442 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:03:54.905449 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:03:54.905464 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:03:54.905983 (+ 519us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:03:54.906058 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:03:54.906079 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:03:54.906225 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 13:03:54.906449 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 13:03:54.906737 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":401,"threads_started":1}
I20260808 13:03:54.911762 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:03:54.916502 10089 raft_consensus.cc:348] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b177da4ed84e2ea3dcbcb3962b500e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 33951 } }
I20260808 13:03:54.916837 10089 raft_consensus.cc:374] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:03:54.916917 10089 raft_consensus.cc:729] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92b177da4ed84e2ea3dcbcb3962b500e, State: Initialized, Role: FOLLOWER
I20260808 13:03:54.917402 10089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b177da4ed84e2ea3dcbcb3962b500e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 33951 } }
I20260808 13:03:54.917595 10089 raft_consensus.cc:388] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:03:54.917693 10089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:03:54.917820 10089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:03:54.920009 10089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b177da4ed84e2ea3dcbcb3962b500e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 33951 } }
I20260808 13:03:54.920574 10089 leader_election.cc:302] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 92b177da4ed84e2ea3dcbcb3962b500e; no voters:
I20260808 13:03:54.921365 10089 leader_election.cc:288] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:03:54.921478 10095 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:03:54.922396 10095 raft_consensus.cc:686] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e [term 1 LEADER]: Becoming Leader. State: Replica: 92b177da4ed84e2ea3dcbcb3962b500e, State: Running, Role: LEADER
I20260808 13:03:54.923004 10095 consensus_queue.cc:237] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b177da4ed84e2ea3dcbcb3962b500e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 33951 } }
I20260808 13:03:54.923816 10089 sys_catalog.cc:564] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:03:54.926322 10097 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 92b177da4ed84e2ea3dcbcb3962b500e. Latest consensus state: current_term: 1 leader_uuid: "92b177da4ed84e2ea3dcbcb3962b500e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b177da4ed84e2ea3dcbcb3962b500e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 33951 } } }
I20260808 13:03:54.926755 10097 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:54.926307 10096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "92b177da4ed84e2ea3dcbcb3962b500e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "92b177da4ed84e2ea3dcbcb3962b500e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 33951 } } }
I20260808 13:03:54.927430 10096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e [sys.catalog]: This master's current role is: LEADER
I20260808 13:03:54.934916 10103 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:03:54.941159 10103 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:03:54.950419 10103 catalog_manager.cc:1347] Generated new cluster ID: 36cd633fc36e484a86402ab46bb9c83a
I20260808 13:03:54.950608 10103 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:03:54.976858 10103 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:03:54.978773 10103 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:03:54.985556 10103 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 92b177da4ed84e2ea3dcbcb3962b500e: Generated new TSK 0
I20260808 13:03:54.986552 10103 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:03:56.278110 10093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:03:55.004277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44888 (local address 127.1.124.190:33951)
0808 13:03:55.004484 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 13:03:55.004491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:03:55.004973 (+ 482us) server_negotiation.cc:407] Connection header received
0808 13:03:55.005238 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:03:55.005246 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:03:55.005328 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:03:55.005518 (+ 190us) server_negotiation.cc:226] Negotiated authn=JWT
0808 13:03:55.006039 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:55.006774 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:55.008256 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:03:55.008724 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:03:55.008867 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:55.009404 (+ 537us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 13:03:56.277264 (+1267860us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 13:03:56.277749 (+ 485us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44888: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 13:03:56.278280 10093 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44888: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 13:03:56.278448 10134 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:03:55.004101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:33951 (local address 127.0.0.1:44888)
0808 13:03:55.004837 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0808 13:03:55.004854 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:03:55.005027 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:03:55.005526 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:03:55.005531 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:03:55.005544 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0808 13:03:55.005845 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:55.005856 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:55.006942 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:55.006949 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:55.008046 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:03:55.008112 (+ 66us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:03:55.008915 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:03:55.008924 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:03:55.009073 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:03:55.009115 (+ 42us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 13:03:56.277605 (+1268490us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 13:03:56.278040 (+ 435us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.1.124.190:33951: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":414,"threads_started":1}
I20260808 13:03:56.279827 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 9978
I20260808 13:03:56.281183 10079 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:03:56.387183 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 9978
2026-08-08T13:03:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (2027 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (435 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (1548 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:03:58 dist-test-slave-t3q3 krb5kdc[10159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:03:58 dist-test-slave-t3q3 krb5kdc[10159](info): set up 2 sockets
Aug 08 13:03:58 dist-test-slave-t3q3 krb5kdc[10159](info): commencing operation
krb5kdc: starting...
W20260808 13:03:59.716568 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.298s user 0.000s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:03:59 dist-test-slave-t3q3 krb5kdc[10159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:03:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:03:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:03:59.972308 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:41077
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:34071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:41077
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:00.314217 10178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:00.314489 10178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:00.314668 10178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:00.321770 10178 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:04:00.321874 10178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:00.321993 10178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:00.322098 10178 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:04:00.322190 10178 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:04:00.331694 10178 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:34071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:41077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:41077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:00.333024 10178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:00.334352 10178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:00.346733 10191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:00.346997 10188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:00.347306 10193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:00.347507 10178 server_base.cc:1034] running on GCE node
I20260808 13:04:00.348441 10178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:00.350046 10178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:00.351344 10178 hybrid_clock.cc:648] HybridClock initialized: now 1786194240351281 us; error 49 us; skew 500 ppm
I20260808 13:04:00.351732 10178 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:00 dist-test-slave-t3q3 krb5kdc[10159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194240, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:00.358431 10178 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:04:00.360142 10178 webserver.cc:460] Webserver started at http://127.1.124.190:34657/ using document root <none> and password file <none>
I20260808 13:04:00.360895 10178 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:00.361071 10178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:00.361418 10178 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:00.364760 10178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "3520927784c84a039e7b80ce007a23a2"
format_stamp: "Formatted at 2026-08-08 13:04:00 on dist-test-slave-t3q3"
server_key: "0a8f14320bb517e52ac59dc22bc9237e"
server_key_iv: "29d1ca8bb08ec9edf85140e84a054cae"
server_key_version: "encryptionkey@0"
I20260808 13:04:00.365649 10178 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "3520927784c84a039e7b80ce007a23a2"
format_stamp: "Formatted at 2026-08-08 13:04:00 on dist-test-slave-t3q3"
server_key: "0a8f14320bb517e52ac59dc22bc9237e"
server_key_iv: "29d1ca8bb08ec9edf85140e84a054cae"
server_key_version: "encryptionkey@0"
I20260808 13:04:00.371258 10178 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260808 13:04:00.374857 10206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:00.376269 10178 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:04:00.376454 10178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "3520927784c84a039e7b80ce007a23a2"
format_stamp: "Formatted at 2026-08-08 13:04:00 on dist-test-slave-t3q3"
server_key: "0a8f14320bb517e52ac59dc22bc9237e"
server_key_iv: "29d1ca8bb08ec9edf85140e84a054cae"
server_key_version: "encryptionkey@0"
I20260808 13:04:00.376660 10178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:00.415479 10178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:00.420092 10178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:00.420426 10178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:00.441027 10178 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:41077
I20260808 13:04:00.441054 10293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:41077 every 8 connection(s)
I20260808 13:04:00.442519 10178 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:04:00.446264 10295 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:00.447719 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 10178
I20260808 13:04:00.447964 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:04:00.448849 1522 external_mini_cluster.cc:1428] Setting key 20a53e18219f3dcf00efb7e801e30954
I20260808 13:04:00.459060 10295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2: Bootstrap starting.
I20260808 13:04:00.463145 10295 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:00.464483 10295 log.cc:824] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2: Log is configured to *not* fsync() on all Append() calls
Aug 08 13:04:00 dist-test-slave-t3q3 krb5kdc[10159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194239, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:04:00.468218 10295 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2: No bootstrap required, opened a new log
I20260808 13:04:00.475060 10299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:00.451097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47878 (local address 127.1.124.190:41077)
0808 13:04:00.452033 (+ 936us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:00.452045 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:00.452174 (+ 129us) server_negotiation.cc:407] Connection header received
0808 13:04:00.454073 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:00.454097 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:00.454480 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:00.454820 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:00.456647 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:00.458166 (+ 1519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:00.459968 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:00.460747 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:00.460910 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:00.466095 (+ 5185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:00.466105 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:00.466126 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:00.466174 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:00.470270 (+ 4096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:00.471021 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:00.471035 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:00.471043 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:00.471274 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:00.471686 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:00.471696 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:00.471700 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:00.472454 (+ 754us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:00.472815 (+ 361us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:00.473200 (+ 385us) server_negotiation.cc:299] Negotiation successful
0808 13:04:00.473658 (+ 458us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":704,"thread_start_us":447,"threads_started":1}
I20260808 13:04:00.478690 10295 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3520927784c84a039e7b80ce007a23a2" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41077 } }
I20260808 13:04:00.479015 10295 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:00.479081 10295 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3520927784c84a039e7b80ce007a23a2, State: Initialized, Role: FOLLOWER
I20260808 13:04:00.479683 10295 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3520927784c84a039e7b80ce007a23a2" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41077 } }
I20260808 13:04:00.480013 10295 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:04:00.480213 10295 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:04:00.480432 10295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:00.482818 10295 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3520927784c84a039e7b80ce007a23a2" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41077 } }
I20260808 13:04:00.483314 10295 leader_election.cc:302] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3520927784c84a039e7b80ce007a23a2; no voters:
I20260808 13:04:00.483901 10295 leader_election.cc:288] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:04:00.484028 10302 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:04:00.484875 10302 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2 [term 1 LEADER]: Becoming Leader. State: Replica: 3520927784c84a039e7b80ce007a23a2, State: Running, Role: LEADER
I20260808 13:04:00.485474 10302 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3520927784c84a039e7b80ce007a23a2" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41077 } }
I20260808 13:04:00.486184 10295 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:04:00.488147 10305 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3520927784c84a039e7b80ce007a23a2. Latest consensus state: current_term: 1 leader_uuid: "3520927784c84a039e7b80ce007a23a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3520927784c84a039e7b80ce007a23a2" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41077 } } }
I20260808 13:04:00.488575 10305 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2 [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:00.489002 10303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3520927784c84a039e7b80ce007a23a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3520927784c84a039e7b80ce007a23a2" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41077 } } }
I20260808 13:04:00.490068 10303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2 [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:00.490135 10312 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:04:00.495872 10312 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:04:00.504303 10312 catalog_manager.cc:1347] Generated new cluster ID: 587b06a86f444369a487a0739555f828
I20260808 13:04:00.504421 10312 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:04:00.517055 10312 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:04:00.518641 10312 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:04:00.533793 10312 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3520927784c84a039e7b80ce007a23a2: Generated new TSK 0
I20260808 13:04:00.534775 10312 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.124.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
I20260808 13:04:00.608969 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:41077
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:34071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:00.967372 10345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:00.967666 10345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:00.967762 10345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:00.975731 10345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:00.975852 10345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:00.975968 10345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:04:00.986064 10345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:34071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:41077
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:00.987521 10345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:00.988936 10345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:01.002774 10362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:01.002822 10354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:01.003046 10351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:01.003721 10345 server_base.cc:1034] running on GCE node
I20260808 13:04:01.004462 10345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:01.005333 10345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:01.006577 10345 hybrid_clock.cc:648] HybridClock initialized: now 1786194241006529 us; error 64 us; skew 500 ppm
I20260808 13:04:01.006928 10345 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:01 dist-test-slave-t3q3 krb5kdc[10159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194241, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:01.010249 10345 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:04:01.011617 10345 webserver.cc:460] Webserver started at http://127.1.124.129:36281/ using document root <none> and password file <none>
I20260808 13:04:01.012355 10345 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:01.012463 10345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:01.012727 10345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:01.015687 10345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "8fc9caa8dff444549daba1b8ece17b73"
format_stamp: "Formatted at 2026-08-08 13:04:01 on dist-test-slave-t3q3"
server_key: "d1de5bf48fe8f0431dff9144681034d4"
server_key_iv: "145a9aa0115fecdd4813345abe05ef62"
server_key_version: "encryptionkey@0"
I20260808 13:04:01.016542 10345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "8fc9caa8dff444549daba1b8ece17b73"
format_stamp: "Formatted at 2026-08-08 13:04:01 on dist-test-slave-t3q3"
server_key: "d1de5bf48fe8f0431dff9144681034d4"
server_key_iv: "145a9aa0115fecdd4813345abe05ef62"
server_key_version: "encryptionkey@0"
I20260808 13:04:01.021777 10345 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:04:01.025345 10380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:01.026798 10345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:04:01.026933 10345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "8fc9caa8dff444549daba1b8ece17b73"
format_stamp: "Formatted at 2026-08-08 13:04:01 on dist-test-slave-t3q3"
server_key: "d1de5bf48fe8f0431dff9144681034d4"
server_key_iv: "145a9aa0115fecdd4813345abe05ef62"
server_key_version: "encryptionkey@0"
I20260808 13:04:01.027122 10345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:01.045120 10345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:01.049451 10345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:01.049696 10345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:01.050637 10345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:01.051950 10345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:01.052024 10345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:01.052178 10345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:01.052232 10345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:01.087656 10345 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:37025
I20260808 13:04:01.087706 10572 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:37025 every 8 connection(s)
I20260808 13:04:01.089039 10345 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:04:01.091455 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 10345
I20260808 13:04:01.091711 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:04:01.092569 1522 external_mini_cluster.cc:1428] Setting key fbf471dea5c2da6937d5bb6e423a1efe
Aug 08 13:04:01 dist-test-slave-t3q3 krb5kdc[10159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194241, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:04:01.109946 10575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:01.091708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:60463 (local address 127.1.124.190:41077)
0808 13:04:01.092253 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:01.092260 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:01.093470 (+ 1210us) server_negotiation.cc:407] Connection header received
0808 13:04:01.095288 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:01.095296 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:01.095392 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:01.095522 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:01.097078 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:01.098226 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:01.099785 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:01.100533 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:01.100674 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:01.104094 (+ 3420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:01.104102 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:01.104106 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:01.104147 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:01.106716 (+ 2569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:01.107479 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:01.107484 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:01.107487 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:01.107644 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:01.108115 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:01.108121 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:01.108127 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:01.108412 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:01.108592 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:01.109305 (+ 713us) server_negotiation.cc:299] Negotiation successful
0808 13:04:01.109550 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":242,"threads_started":1}
I20260808 13:04:01.110956 10576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:01.092334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:41077 (local address 127.1.124.129:60463)
0808 13:04:01.093308 (+ 974us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:01.093354 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:01.095005 (+ 1651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:01.095720 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:01.095728 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:01.096337 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:01.096903 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:01.096921 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:01.098393 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:01.098396 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:01.099637 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:01.099649 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:01.100740 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:01.100744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:01.100935 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:01.101640 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:01.101667 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:01.103849 (+ 2182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:01.106872 (+ 3023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:01.106880 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:01.106897 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:01.107320 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:01.107768 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:01.107773 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:01.107777 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:01.107972 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:01.108552 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:01.108560 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:01.108917 (+ 357us) client_negotiation.cc:769] Sending connection context
0808 13:04:01.109208 (+ 291us) client_negotiation.cc:240] Negotiation successful
0808 13:04:01.109763 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":368,"threads_started":1}
I20260808 13:04:01.112567 10573 heartbeater.cc:347] Connected to a master server at 127.1.124.190:41077
I20260808 13:04:01.112902 10573 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:01.113708 10573 heartbeater.cc:511] Master 127.1.124.190:41077 requested a full tablet report, sending...
I20260808 13:04:01.115923 10229 ts_manager.cc:194] Registered new tserver with Master: 8fc9caa8dff444549daba1b8ece17b73 (127.1.124.129:37025)
I20260808 13:04:01.117415 10229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:60463
WARNING: no policy specified for kudu/127.1.124.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
I20260808 13:04:01.151722 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:41077
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:34071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:01.507620 10583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:01.507968 10583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:01.508177 10583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:01.515545 10583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:01.515707 10583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:01.515897 10583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:04:01.526341 10583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:34071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:41077
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:01.527891 10583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:01.529486 10583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:01.544433 10589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:01.544735 10591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:01.544764 10593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:01.545420 10583 server_base.cc:1034] running on GCE node
I20260808 13:04:01.546044 10583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:01.547390 10583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:01.548626 10583 hybrid_clock.cc:648] HybridClock initialized: now 1786194241548571 us; error 67 us; skew 500 ppm
I20260808 13:04:01.548959 10583 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:01 dist-test-slave-t3q3 krb5kdc[10159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194241, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:01.552419 10583 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:04:01.553987 10583 webserver.cc:460] Webserver started at http://127.1.124.130:44627/ using document root <none> and password file <none>
I20260808 13:04:01.554695 10583 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:01.554847 10583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:01.555156 10583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:01.558277 10583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "c2c2e08d1e8643ad9c80a1086dc98ceb"
format_stamp: "Formatted at 2026-08-08 13:04:01 on dist-test-slave-t3q3"
server_key: "256ff75a000bda84a0f8a415bba1b390"
server_key_iv: "adab928b53e13d127ef8b219b68f925e"
server_key_version: "encryptionkey@0"
I20260808 13:04:01.559118 10583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "c2c2e08d1e8643ad9c80a1086dc98ceb"
format_stamp: "Formatted at 2026-08-08 13:04:01 on dist-test-slave-t3q3"
server_key: "256ff75a000bda84a0f8a415bba1b390"
server_key_iv: "adab928b53e13d127ef8b219b68f925e"
server_key_version: "encryptionkey@0"
I20260808 13:04:01.564543 10583 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 13:04:01.568133 10607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:01.569408 10583 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:04:01.569603 10583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "c2c2e08d1e8643ad9c80a1086dc98ceb"
format_stamp: "Formatted at 2026-08-08 13:04:01 on dist-test-slave-t3q3"
server_key: "256ff75a000bda84a0f8a415bba1b390"
server_key_iv: "adab928b53e13d127ef8b219b68f925e"
server_key_version: "encryptionkey@0"
I20260808 13:04:01.569823 10583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:01.586350 10583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:01.590744 10583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:01.591046 10583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:01.592196 10583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:01.593659 10583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:01.593765 10583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:01.593917 10583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:01.593988 10583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:01.629159 10583 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:44443
I20260808 13:04:01.629249 10787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:44443 every 8 connection(s)
I20260808 13:04:01.630541 10583 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:04:01.639002 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 10583
I20260808 13:04:01.639202 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:04:01.639683 1522 external_mini_cluster.cc:1428] Setting key 0f45dd702a21f0ae8ad28e3f918b99ba
Aug 08 13:04:01 dist-test-slave-t3q3 krb5kdc[10159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194241, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:04:01.651983 10793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:01.633651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:39657 (local address 127.1.124.190:41077)
0808 13:04:01.634138 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:01.634143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:01.635133 (+ 990us) server_negotiation.cc:407] Connection header received
0808 13:04:01.637047 (+ 1914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:01.637052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:01.637134 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:01.637239 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:01.638736 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:01.639691 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:01.641699 (+ 2008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:01.642156 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:01.642302 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:01.645948 (+ 3646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:01.645953 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:01.645956 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:01.645990 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:01.648771 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:01.649564 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:01.649569 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:01.649573 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:01.649705 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:01.650170 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:01.650179 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:01.650181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:01.650446 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:01.650619 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:01.651343 (+ 724us) server_negotiation.cc:299] Negotiation successful
0808 13:04:01.651581 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":254,"threads_started":1}
I20260808 13:04:01.653047 10795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:01.634242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:41077 (local address 127.1.124.130:39657)
0808 13:04:01.634972 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:01.635009 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:01.636772 (+ 1763us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:01.637445 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:01.637452 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:01.637976 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:01.638523 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:01.638551 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:01.639900 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:01.639905 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:01.641519 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:01.641532 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:01.642394 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:01.642403 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:01.642650 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:01.643370 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:01.643397 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:01.645708 (+ 2311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:01.648940 (+ 3232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:01.648950 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:01.648967 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:01.649418 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:01.649833 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:01.649837 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:01.649841 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:01.650041 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:01.650591 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:01.650600 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:01.650956 (+ 356us) client_negotiation.cc:769] Sending connection context
0808 13:04:01.651251 (+ 295us) client_negotiation.cc:240] Negotiation successful
0808 13:04:01.651813 (+ 562us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":295,"threads_started":1}
I20260808 13:04:01.654623 10789 heartbeater.cc:347] Connected to a master server at 127.1.124.190:41077
I20260808 13:04:01.654939 10789 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:01.655697 10789 heartbeater.cc:511] Master 127.1.124.190:41077 requested a full tablet report, sending...
I20260808 13:04:01.657349 10229 ts_manager.cc:194] Registered new tserver with Master: c2c2e08d1e8643ad9c80a1086dc98ceb (127.1.124.130:44443)
I20260808 13:04:01.658644 10229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:39657
WARNING: no policy specified for kudu/127.1.124.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
I20260808 13:04:01.700194 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:41077
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:34071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:02.041731 10799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:02.042042 10799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:02.042138 10799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:02.049264 10799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:02.049384 10799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:02.049499 10799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:04:02.059527 10799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:34071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:41077
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:02.060881 10799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:02.062224 10799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:02.075763 10805 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:02.075827 10809 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:02.075856 10804 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:02.077461 10799 server_base.cc:1034] running on GCE node
I20260808 13:04:02.078150 10799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:02.079208 10799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:02.080498 10799 hybrid_clock.cc:648] HybridClock initialized: now 1786194242080430 us; error 48 us; skew 500 ppm
I20260808 13:04:02.080860 10799 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:02 dist-test-slave-t3q3 krb5kdc[10159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194242, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:02.084381 10799 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:04:02.086046 10799 webserver.cc:460] Webserver started at http://127.1.124.131:35567/ using document root <none> and password file <none>
I20260808 13:04:02.086869 10799 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:02.087026 10799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:02.087425 10799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:02.090601 10799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "63c4b93201eb43acbe9602847bed67b2"
format_stamp: "Formatted at 2026-08-08 13:04:02 on dist-test-slave-t3q3"
server_key: "91de8e60d9b0253deb89887f2c857a02"
server_key_iv: "af68fac5bd4770e57e01d6fdf8ea0608"
server_key_version: "encryptionkey@0"
I20260808 13:04:02.091562 10799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "63c4b93201eb43acbe9602847bed67b2"
format_stamp: "Formatted at 2026-08-08 13:04:02 on dist-test-slave-t3q3"
server_key: "91de8e60d9b0253deb89887f2c857a02"
server_key_iv: "af68fac5bd4770e57e01d6fdf8ea0608"
server_key_version: "encryptionkey@0"
I20260808 13:04:02.097254 10799 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 13:04:02.101480 10820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:02.102859 10799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:04:02.103049 10799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "63c4b93201eb43acbe9602847bed67b2"
format_stamp: "Formatted at 2026-08-08 13:04:02 on dist-test-slave-t3q3"
server_key: "91de8e60d9b0253deb89887f2c857a02"
server_key_iv: "af68fac5bd4770e57e01d6fdf8ea0608"
server_key_version: "encryptionkey@0"
I20260808 13:04:02.103266 10799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:02.117254 10799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:02.120498 10573 heartbeater.cc:503] Master 127.1.124.190:41077 was elected leader, sending a full tablet report...
I20260808 13:04:02.121765 10799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:02.122090 10799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:02.123215 10799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:02.124665 10799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:02.124742 10799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:02.124910 10799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:02.124989 10799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:02.158461 10799 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:37843
I20260808 13:04:02.158517 11000 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:37843 every 8 connection(s)
I20260808 13:04:02.159786 10799 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:04:02.166857 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 10799
I20260808 13:04:02.167081 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:04:02.167680 1522 external_mini_cluster.cc:1428] Setting key bbf4a44af39a0f17c1a3a25506af5028
Aug 08 13:04:02 dist-test-slave-t3q3 krb5kdc[10159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194242, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:04:02.181576 11004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:02.162566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:51201 (local address 127.1.124.190:41077)
0808 13:04:02.163056 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:02.163061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:02.164109 (+ 1048us) server_negotiation.cc:407] Connection header received
0808 13:04:02.166362 (+ 2253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:02.166370 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:02.166495 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:02.166614 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:02.168762 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:02.169807 (+ 1045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:02.171404 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:02.171813 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:02.171958 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:02.175709 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:02.175714 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:02.175718 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:02.175748 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:02.178379 (+ 2631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:02.179136 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:02.179145 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:02.179147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:02.179262 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:02.179733 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:02.179736 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:02.179739 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:02.180014 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:02.180228 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:02.180945 (+ 717us) server_negotiation.cc:299] Negotiation successful
0808 13:04:02.181172 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":228,"threads_started":1}
I20260808 13:04:02.182638 11006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:02.163131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:41077 (local address 127.1.124.131:51201)
0808 13:04:02.163895 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:02.163939 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:02.165821 (+ 1882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:02.167072 (+ 1251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:02.167085 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:02.167815 (+ 730us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:02.168584 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:02.168602 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:02.169970 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:02.169975 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:02.171239 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:02.171250 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:02.172104 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:02.172113 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:02.172337 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:02.173070 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:02.173092 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:02.175486 (+ 2394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:02.178533 (+ 3047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:02.178546 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:02.178564 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:02.178995 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:02.179380 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:02.179385 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:02.179389 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:02.179580 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:02.180219 (+ 639us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:02.180227 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:02.180566 (+ 339us) client_negotiation.cc:769] Sending connection context
0808 13:04:02.180830 (+ 264us) client_negotiation.cc:240] Negotiation successful
0808 13:04:02.181382 (+ 552us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":350,"threads_started":1}
I20260808 13:04:02.184163 11001 heartbeater.cc:347] Connected to a master server at 127.1.124.190:41077
I20260808 13:04:02.184489 11001 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:02.185230 11001 heartbeater.cc:511] Master 127.1.124.190:41077 requested a full tablet report, sending...
I20260808 13:04:02.186766 10229 ts_manager.cc:194] Registered new tserver with Master: 63c4b93201eb43acbe9602847bed67b2 (127.1.124.131:37843)
I20260808 13:04:02.187661 10229 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:51201
I20260808 13:04:02.195897 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/test-admin.keytab.
Aug 08 13:04:02 dist-test-slave-t3q3 krb5kdc[10159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:02.230654 1522 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 13:04:02.236241 11012 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:02.233591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50776 (local address 127.1.124.129:37025)
0808 13:04:02.234301 (+ 710us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:02.234309 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:02.234337 (+ 28us) server_negotiation.cc:407] Connection header received
0808 13:04:02.234399 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:02.234406 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:02.234633 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:02.234772 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:02.235719 (+ 947us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50776: BlockingRecv error: recv got EOF from 127.0.0.1:50776 (error 108)
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":367,"threads_started":1}
W20260808 13:04:02.236374 1522 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/krb5cc' of type FILE
Aug 08 13:04:02 dist-test-slave-t3q3 krb5kdc[10159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:02.238022 1522 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 13:04:02 dist-test-slave-t3q3 krb5kdc[10159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:02.239424 1522 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 13:04:02 dist-test-slave-t3q3 krb5kdc[10159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.129@KRBTEST.COM
I20260808 13:04:02.256153 11012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:02.241228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50780 (local address 127.1.124.129:37025)
0808 13:04:02.241471 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:02.241479 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:02.241745 (+ 266us) server_negotiation.cc:407] Connection header received
0808 13:04:02.241957 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:02.241961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:02.242023 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:02.242138 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:02.243238 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:02.244361 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:02.245674 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:02.246266 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:02.246416 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:02.250204 (+ 3788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:02.250217 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:02.250232 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:02.250267 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:02.253253 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:02.253965 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:02.253973 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:02.253975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:02.254097 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:02.254479 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:02.254484 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:02.254487 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:02.254870 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:02.255182 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:02.255529 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 13:04:02.255809 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
W20260808 13:04:02.262997 11012 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:02.260533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50782 (local address 127.1.124.129:37025)
0808 13:04:02.260786 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:02.260795 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:02.261087 (+ 292us) server_negotiation.cc:407] Connection header received
0808 13:04:02.261326 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:02.261334 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:02.261446 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:02.261552 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:02.262736 (+ 1184us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50782: BlockingRecv error: recv got EOF from 127.0.0.1:50782 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260808 13:04:02.263237 1522 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/krb5cc' of type FILE
Aug 08 13:04:02 dist-test-slave-t3q3 krb5kdc[10159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:02.265195 1522 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 13:04:02 dist-test-slave-t3q3 krb5kdc[10159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:02.266608 1522 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 13:04:02 dist-test-slave-t3q3 krb5kdc[10159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.129@KRBTEST.COM
I20260808 13:04:02.282496 11012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:02.268104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50786 (local address 127.1.124.129:37025)
0808 13:04:02.268277 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:02.268283 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:02.268404 (+ 121us) server_negotiation.cc:407] Connection header received
0808 13:04:02.268535 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:02.268539 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:02.268593 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:02.268675 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:02.269827 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:02.270580 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:02.271836 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:02.272384 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:02.272506 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:02.276846 (+ 4340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:02.276851 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:02.276854 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:02.276907 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:02.279608 (+ 2701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:02.280468 (+ 860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:02.280472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:02.280474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:02.280611 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:02.280994 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:02.280998 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:02.281001 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:02.281388 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:02.281687 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:02.281984 (+ 297us) server_negotiation.cc:299] Negotiation successful
0808 13:04:02.282209 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 13:04:02.287633 11012 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:02.285521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50800 (local address 127.1.124.129:37025)
0808 13:04:02.285746 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:02.285753 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:02.285997 (+ 244us) server_negotiation.cc:407] Connection header received
0808 13:04:02.286272 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:02.286276 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:02.286350 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:02.286436 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:02.287392 (+ 956us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50800: BlockingRecv error: recv got EOF from 127.0.0.1:50800 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260808 13:04:02.287875 1522 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786194185613124-1522-0/krb5kdc/krb5cc' of type FILE
Aug 08 13:04:02 dist-test-slave-t3q3 krb5kdc[10159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:02.289587 1522 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 13:04:02 dist-test-slave-t3q3 krb5kdc[10159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:02.291129 1522 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 13:04:02 dist-test-slave-t3q3 krb5kdc[10159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.129@KRBTEST.COM
I20260808 13:04:02.307103 11012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:02.292803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50814 (local address 127.1.124.129:37025)
0808 13:04:02.293043 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:02.293049 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:02.293317 (+ 268us) server_negotiation.cc:407] Connection header received
0808 13:04:02.293439 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:02.293443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:02.293501 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:02.293607 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:02.294641 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:02.295401 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:02.296809 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:02.297291 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:02.297441 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:02.301598 (+ 4157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:02.301604 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:02.301607 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:02.301646 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:02.304376 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:02.305052 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:02.305057 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:02.305089 (+ 32us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:02.305250 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:02.305655 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:02.305659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:02.305661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:02.305932 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:02.306182 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:02.306483 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 13:04:02.306664 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 13:04:02.309092 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 10345
I20260808 13:04:02.322366 11012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:02.310250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50820 (local address 127.1.124.129:37025)
0808 13:04:02.310504 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:02.310518 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:02.310662 (+ 144us) server_negotiation.cc:407] Connection header received
0808 13:04:02.310878 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:02.310883 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:02.310964 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:02.311073 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:02.312135 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:02.312863 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:02.314130 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:02.314559 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:02.314712 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:02.316971 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:02.316982 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:02.316985 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:02.317018 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:02.319563 (+ 2545us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:02.320365 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:02.320373 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:02.320378 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:02.320532 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:02.320913 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:02.320918 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:02.320922 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:02.321205 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:02.321495 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:02.321804 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 13:04:02.322038 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 13:04:02.322992 10566 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:02.410215 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 10345
I20260808 13:04:02.434971 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 10583
Aug 08 13:04:02 dist-test-slave-t3q3 krb5kdc[10159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.130@KRBTEST.COM
I20260808 13:04:02.452528 11027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:02.435988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44242 (local address 127.1.124.130:44443)
0808 13:04:02.436613 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:02.436622 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:02.436649 (+ 27us) server_negotiation.cc:407] Connection header received
0808 13:04:02.436722 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:02.436729 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:02.436939 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:02.437082 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:02.438090 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:02.439160 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:02.440648 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:02.441248 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:02.441355 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:02.446119 (+ 4764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:02.446133 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:02.446148 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:02.446186 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:02.449326 (+ 3140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:02.450135 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:02.450143 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:02.450146 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:02.450296 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:02.450794 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:02.450799 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:02.450801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:02.451212 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:02.451444 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:02.451821 (+ 377us) server_negotiation.cc:299] Negotiation successful
0808 13:04:02.452195 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":351,"threads_started":1}
I20260808 13:04:02.454054 10776 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:02.537941 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 10583
I20260808 13:04:02.562491 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 10799
Aug 08 13:04:02 dist-test-slave-t3q3 krb5kdc[10159](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194242, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.131@KRBTEST.COM
I20260808 13:04:02.580261 11031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:02.563717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57092 (local address 127.1.124.131:37843)
0808 13:04:02.564594 (+ 877us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:02.564601 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:02.564632 (+ 31us) server_negotiation.cc:407] Connection header received
0808 13:04:02.564694 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:02.564701 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:02.564910 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:02.565099 (+ 189us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:02.566233 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:02.567335 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:02.568799 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:02.569374 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:02.569489 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:02.573915 (+ 4426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:02.573930 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:02.573946 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:02.574007 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:02.577194 (+ 3187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:02.577947 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:02.577955 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:02.577957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:02.578135 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:02.578511 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:02.578515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:02.578517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:02.578901 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:02.579215 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:02.579496 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 13:04:02.579792 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":531,"threads_started":1}
I20260808 13:04:02.581748 10991 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:02.666061 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 10799
I20260808 13:04:02.689991 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 10178
I20260808 13:04:02.691296 10280 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:02.800343 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 10178
2026-08-08T13:04:02Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4430 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:04:02 dist-test-slave-t3q3 krb5kdc[11037](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:04:02 dist-test-slave-t3q3 krb5kdc[11037](info): set up 2 sockets
Aug 08 13:04:02 dist-test-slave-t3q3 krb5kdc[11037](info): commencing operation
krb5kdc: starting...
W20260808 13:04:04.144474 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.000s sys 0.033s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:04:04 dist-test-slave-t3q3 krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:04:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:04:04Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.190.keytab.
Entry for principal oryx/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.190.keytab.
I20260808 13:04:04.412485 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:39241
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.190.keytab
--principal=oryx/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:41301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:39241
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:04.768147 11055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:04.768539 11055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:04.768668 11055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:04.776026 11055 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:04:04.776207 11055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:04.776259 11055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:04.776310 11055 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:04:04.776357 11055 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:04:04.786540 11055 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:41301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:39241
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.190.keytab
--principal=oryx/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:39241
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:04.787905 11055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:04.789391 11055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:04.801928 11063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:04.801908 11067 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:04.801923 11065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:04.803143 11055 server_base.cc:1034] running on GCE node
I20260808 13:04:04.804000 11055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:04.805512 11055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:04.806731 11055 hybrid_clock.cc:648] HybridClock initialized: now 1786194244806694 us; error 34 us; skew 500 ppm
I20260808 13:04:04.807060 11055 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:04 dist-test-slave-t3q3 krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194244, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:04.810463 11055 init.cc:377] Logged in from keytab as oryx/127.1.124.190@KRBTEST.COM (short username oryx)
I20260808 13:04:04.812132 11055 webserver.cc:460] Webserver started at http://127.1.124.190:40843/ using document root <none> and password file <none>
I20260808 13:04:04.812867 11055 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:04.813026 11055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:04.813351 11055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:04.816526 11055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "cdbf0bd6947747b9a2f3229974732e17"
format_stamp: "Formatted at 2026-08-08 13:04:04 on dist-test-slave-t3q3"
server_key: "92915cdf6ee9049d74c549a55aa83d74"
server_key_iv: "7b72b7ec271ec0f2f391196f0482ed4d"
server_key_version: "encryptionkey@0"
I20260808 13:04:04.817358 11055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "cdbf0bd6947747b9a2f3229974732e17"
format_stamp: "Formatted at 2026-08-08 13:04:04 on dist-test-slave-t3q3"
server_key: "92915cdf6ee9049d74c549a55aa83d74"
server_key_iv: "7b72b7ec271ec0f2f391196f0482ed4d"
server_key_version: "encryptionkey@0"
I20260808 13:04:04.822985 11055 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260808 13:04:04.826891 11077 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:04.828562 11055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:04:04.828773 11055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "cdbf0bd6947747b9a2f3229974732e17"
format_stamp: "Formatted at 2026-08-08 13:04:04 on dist-test-slave-t3q3"
server_key: "92915cdf6ee9049d74c549a55aa83d74"
server_key_iv: "7b72b7ec271ec0f2f391196f0482ed4d"
server_key_version: "encryptionkey@0"
I20260808 13:04:04.829000 11055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:04.857199 11055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:04.861557 11055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:04.861871 11055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:04.881172 11055 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:39241
I20260808 13:04:04.881222 11180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:39241 every 8 connection(s)
I20260808 13:04:04.882596 11055 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:04:04.886328 11181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:04.890981 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 11055
I20260808 13:04:04.891263 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:04:04.891953 1522 external_mini_cluster.cc:1428] Setting key b8bb76f544c32eb75eef638f7082175e
I20260808 13:04:04.898829 11181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17: Bootstrap starting.
I20260808 13:04:04.902686 11181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:04.903728 11181 log.cc:824] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17: Log is configured to *not* fsync() on all Append() calls
I20260808 13:04:04.906804 11181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17: No bootstrap required, opened a new log
Aug 08 13:04:04 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM
I20260808 13:04:04.916419 11181 raft_consensus.cc:348] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } }
I20260808 13:04:04.916796 11181 raft_consensus.cc:374] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:04.916935 11181 raft_consensus.cc:729] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cdbf0bd6947747b9a2f3229974732e17, State: Initialized, Role: FOLLOWER
I20260808 13:04:04.917519 11181 consensus_queue.cc:260] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } }
I20260808 13:04:04.917826 11181 raft_consensus.cc:388] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:04:04.917856 11188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:04.894428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57178 (local address 127.1.124.190:39241)
0808 13:04:04.895378 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:04.895395 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:04.895453 (+ 58us) server_negotiation.cc:407] Connection header received
0808 13:04:04.897887 (+ 2434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:04.897916 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:04.898419 (+ 503us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:04.898881 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:04.900484 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:04.902496 (+ 2012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:04.903856 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:04.904822 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:04.904960 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:04.909548 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:04.909557 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:04.909575 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:04.909616 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:04.912995 (+ 3379us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:04.913940 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:04.913947 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:04.913953 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:04.914165 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:04.914601 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:04.914605 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:04.914607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:04.915197 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:04.915434 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:04.915874 (+ 440us) server_negotiation.cc:299] Negotiation successful
0808 13:04:04.916435 (+ 561us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":641,"thread_start_us":413,"threads_started":1}
I20260808 13:04:04.918007 11181 raft_consensus.cc:482] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:04:04.918253 11181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:04.920718 11181 raft_consensus.cc:504] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } }
I20260808 13:04:04.921236 11181 leader_election.cc:302] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cdbf0bd6947747b9a2f3229974732e17; no voters:
I20260808 13:04:04.921780 11181 leader_election.cc:288] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:04:04.921950 11190 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:04:04.922652 11190 raft_consensus.cc:686] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [term 1 LEADER]: Becoming Leader. State: Replica: cdbf0bd6947747b9a2f3229974732e17, State: Running, Role: LEADER
I20260808 13:04:04.923347 11190 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } }
I20260808 13:04:04.924789 11181 sys_catalog.cc:564] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:04:04.926267 11191 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdbf0bd6947747b9a2f3229974732e17. Latest consensus state: current_term: 1 leader_uuid: "cdbf0bd6947747b9a2f3229974732e17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } }
I20260808 13:04:04.926293 11192 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cdbf0bd6947747b9a2f3229974732e17" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } }
I20260808 13:04:04.926708 11191 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:04.926721 11192 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:04.929000 11201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:04:04.933745 11201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:04:04.942158 11201 catalog_manager.cc:1347] Generated new cluster ID: 530b8a733f4e4536b5d2220088c480e1
I20260808 13:04:04.942279 11201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:04:04.967223 11201 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:04:04.968725 11201 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:04:04.986827 11201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17: Generated new TSK 0
I20260808 13:04:04.987985 11201 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.124.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.124.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.124.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.129.keytab.
Entry for principal oryx/127.1.124.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.129.keytab.
I20260808 13:04:05.064572 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:39241
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.129.keytab
--principal=oryx/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:41301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:05.415702 11219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:05.415987 11219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:05.416147 11219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:05.423583 11219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:05.423700 11219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:05.423815 11219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:04:05.434270 11219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:41301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.129.keytab
--principal=oryx/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:39241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:05.435595 11219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:05.437155 11219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:05.451133 11231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:05.451197 11227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:05.451102 11225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:05.451941 11219 server_base.cc:1034] running on GCE node
I20260808 13:04:05.452696 11219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:05.453732 11219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:05.454985 11219 hybrid_clock.cc:648] HybridClock initialized: now 1786194245454934 us; error 73 us; skew 500 ppm
I20260808 13:04:05.455381 11219 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:05 dist-test-slave-t3q3 krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194245, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:05.464452 11219 init.cc:377] Logged in from keytab as oryx/127.1.124.129@KRBTEST.COM (short username oryx)
I20260808 13:04:05.466111 11219 webserver.cc:460] Webserver started at http://127.1.124.129:42799/ using document root <none> and password file <none>
I20260808 13:04:05.466854 11219 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:05.467020 11219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:05.467343 11219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:05.470671 11219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "7a9c51158a034c87bc80b34266e50afe"
format_stamp: "Formatted at 2026-08-08 13:04:05 on dist-test-slave-t3q3"
server_key: "89c2a0266efd496f41f15be57e4d8c7b"
server_key_iv: "9e55978342bf49d8f59347bda5b529a4"
server_key_version: "encryptionkey@0"
I20260808 13:04:05.471565 11219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "7a9c51158a034c87bc80b34266e50afe"
format_stamp: "Formatted at 2026-08-08 13:04:05 on dist-test-slave-t3q3"
server_key: "89c2a0266efd496f41f15be57e4d8c7b"
server_key_iv: "9e55978342bf49d8f59347bda5b529a4"
server_key_version: "encryptionkey@0"
I20260808 13:04:05.477067 11219 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 13:04:05.480854 11241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:05.482391 11219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:04:05.482580 11219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "7a9c51158a034c87bc80b34266e50afe"
format_stamp: "Formatted at 2026-08-08 13:04:05 on dist-test-slave-t3q3"
server_key: "89c2a0266efd496f41f15be57e4d8c7b"
server_key_iv: "9e55978342bf49d8f59347bda5b529a4"
server_key_version: "encryptionkey@0"
I20260808 13:04:05.482812 11219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:05.508440 11219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:05.512765 11219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:05.513106 11219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:05.514196 11219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:05.515628 11219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:05.515738 11219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:05.515897 11219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:05.515973 11219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:05.552846 11219 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:35393
I20260808 13:04:05.552949 11433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:35393 every 8 connection(s)
I20260808 13:04:05.554227 11219 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:04:05.563556 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 11219
I20260808 13:04:05.563828 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:04:05.564574 1522 external_mini_cluster.cc:1428] Setting key a3e88a0c44d763456bdb71cf5467a651
Aug 08 13:04:05 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194245, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM
I20260808 13:04:05.575443 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:05.557079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:50397 (local address 127.1.124.190:39241)
0808 13:04:05.557619 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:05.557638 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:05.558557 (+ 919us) server_negotiation.cc:407] Connection header received
0808 13:04:05.560451 (+ 1894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:05.560457 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:05.560546 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:05.560669 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:05.562168 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:05.562915 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:05.564681 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:05.565089 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:05.565261 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:05.569389 (+ 4128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:05.569394 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:05.569397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:05.569428 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:05.572028 (+ 2600us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:05.573086 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:05.573090 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:05.573092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:05.573223 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:05.573719 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:05.573722 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:05.573724 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:05.573969 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:05.574077 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:05.575004 (+ 927us) server_negotiation.cc:299] Negotiation successful
0808 13:04:05.575187 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":266,"threads_started":1}
I20260808 13:04:05.577006 11438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:05.557660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.129:50397)
0808 13:04:05.558388 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:05.558433 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:05.560187 (+ 1754us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:05.560911 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:05.560918 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:05.561415 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:05.561987 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:05.562004 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:05.563081 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:05.563086 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:05.564519 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:05.564531 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:05.565370 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:05.565378 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:05.565610 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:05.566330 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:05.566352 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:05.569171 (+ 2819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:05.572224 (+ 3053us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:05.572235 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:05.572255 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:05.572917 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:05.573336 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:05.573344 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:05.573350 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:05.573603 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:05.574108 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:05.574118 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:05.574591 (+ 473us) client_negotiation.cc:769] Sending connection context
0808 13:04:05.574910 (+ 319us) client_negotiation.cc:240] Negotiation successful
0808 13:04:05.575508 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":278,"threads_started":1}
I20260808 13:04:05.578789 11434 heartbeater.cc:347] Connected to a master server at 127.1.124.190:39241
I20260808 13:04:05.579142 11434 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:05.579875 11434 heartbeater.cc:511] Master 127.1.124.190:39241 requested a full tablet report, sending...
I20260808 13:04:05.582204 11117 ts_manager.cc:194] Registered new tserver with Master: 7a9c51158a034c87bc80b34266e50afe (127.1.124.129:35393)
I20260808 13:04:05.583693 11117 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:50397
WARNING: no policy specified for oryx/127.1.124.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.124.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.124.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.130.keytab.
Entry for principal oryx/127.1.124.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.130.keytab.
I20260808 13:04:05.624377 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:39241
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.130.keytab
--principal=oryx/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:41301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:05.976773 11443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:05.977134 11443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:05.977312 11443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:05.984974 11443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:05.985149 11443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:05.985327 11443 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:04:05.995769 11443 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:41301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.130.keytab
--principal=oryx/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:39241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:05.997216 11443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:05.998752 11443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:06.012354 11453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:06.012384 11454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:06.012624 11456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:06.013650 11443 server_base.cc:1034] running on GCE node
I20260808 13:04:06.014331 11443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:06.015304 11443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:06.016574 11443 hybrid_clock.cc:648] HybridClock initialized: now 1786194246016532 us; error 54 us; skew 500 ppm
I20260808 13:04:06.016940 11443 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:06 dist-test-slave-t3q3 krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194246, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:06.020331 11443 init.cc:377] Logged in from keytab as oryx/127.1.124.130@KRBTEST.COM (short username oryx)
I20260808 13:04:06.021852 11443 webserver.cc:460] Webserver started at http://127.1.124.130:33617/ using document root <none> and password file <none>
I20260808 13:04:06.022576 11443 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:06.022743 11443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:06.023069 11443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:06.026111 11443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "001d8fb8a10e470b8ea6e80f680c6ebb"
format_stamp: "Formatted at 2026-08-08 13:04:06 on dist-test-slave-t3q3"
server_key: "97e4d5ba116a97b50d79f70c92966c7f"
server_key_iv: "46072441219b7f5315cf4b0dad59f0a3"
server_key_version: "encryptionkey@0"
I20260808 13:04:06.026899 11443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "001d8fb8a10e470b8ea6e80f680c6ebb"
format_stamp: "Formatted at 2026-08-08 13:04:06 on dist-test-slave-t3q3"
server_key: "97e4d5ba116a97b50d79f70c92966c7f"
server_key_iv: "46072441219b7f5315cf4b0dad59f0a3"
server_key_version: "encryptionkey@0"
I20260808 13:04:06.032701 11443 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 13:04:06.036548 11467 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:06.038172 11443 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 13:04:06.038347 11443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "001d8fb8a10e470b8ea6e80f680c6ebb"
format_stamp: "Formatted at 2026-08-08 13:04:06 on dist-test-slave-t3q3"
server_key: "97e4d5ba116a97b50d79f70c92966c7f"
server_key_iv: "46072441219b7f5315cf4b0dad59f0a3"
server_key_version: "encryptionkey@0"
I20260808 13:04:06.038582 11443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:06.056172 11443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:06.060252 11443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:06.060552 11443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:06.061509 11443 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:06.062903 11443 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:06.063023 11443 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:06.063187 11443 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:06.063265 11443 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:06.097254 11443 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:45593
I20260808 13:04:06.097302 11635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:45593 every 8 connection(s)
I20260808 13:04:06.098590 11443 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:04:06.102080 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 11443
I20260808 13:04:06.102308 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:04:06.102780 1522 external_mini_cluster.cc:1428] Setting key bdceff903b40bd9f2753dd26b8bc4655
Aug 08 13:04:06 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194246, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM
I20260808 13:04:06.121369 11639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:06.101678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:36107 (local address 127.1.124.190:39241)
0808 13:04:06.102208 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:06.102216 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:06.103416 (+ 1200us) server_negotiation.cc:407] Connection header received
0808 13:04:06.105578 (+ 2162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:06.105587 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:06.105687 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:06.105807 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:06.107331 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:06.108509 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:06.110144 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:06.110907 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:06.111056 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:06.114928 (+ 3872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:06.114934 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:06.114937 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:06.114972 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:06.118005 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:06.118839 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:06.118846 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:06.118849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:06.119000 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:06.119455 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:06.119461 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:06.119464 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:06.119740 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:06.119906 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:06.120688 (+ 782us) server_negotiation.cc:299] Negotiation successful
0808 13:04:06.120943 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":255,"threads_started":1}
I20260808 13:04:06.122527 11641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:06.102343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.130:36107)
0808 13:04:06.103233 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:06.103280 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:06.105297 (+ 2017us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:06.106016 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:06.106024 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:06.106573 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:06.107149 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:06.107167 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:06.108678 (+ 1511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:06.108682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:06.109982 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:06.109996 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:06.111125 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:06.111129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:06.111335 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:06.112109 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:06.112142 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:06.114669 (+ 2527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:06.118209 (+ 3540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:06.118219 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:06.118238 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:06.118703 (+ 465us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:06.119122 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:06.119130 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:06.119135 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:06.119330 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:06.119881 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:06.119890 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:06.120324 (+ 434us) client_negotiation.cc:769] Sending connection context
0808 13:04:06.120625 (+ 301us) client_negotiation.cc:240] Negotiation successful
0808 13:04:06.121246 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":364,"threads_started":1}
I20260808 13:04:06.124212 11636 heartbeater.cc:347] Connected to a master server at 127.1.124.190:39241
I20260808 13:04:06.124528 11636 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:06.125317 11636 heartbeater.cc:511] Master 127.1.124.190:39241 requested a full tablet report, sending...
I20260808 13:04:06.126991 11117 ts_manager.cc:194] Registered new tserver with Master: 001d8fb8a10e470b8ea6e80f680c6ebb (127.1.124.130:45593)
I20260808 13:04:06.127840 11117 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:36107
WARNING: no policy specified for oryx/127.1.124.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.124.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.124.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.131.keytab.
Entry for principal oryx/127.1.124.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.131.keytab.
I20260808 13:04:06.164327 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:39241
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.131.keytab
--principal=oryx/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:41301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:06.517185 11644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:06.517544 11644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:06.517648 11644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:06.525138 11644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:06.525264 11644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:06.525379 11644 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:04:06.535835 11644 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:41301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.131.keytab
--principal=oryx/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:39241
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:06.537297 11644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:06.538671 11644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:06.553118 11653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:06.553400 11655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:06.553735 11650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:06.557885 11644 server_base.cc:1034] running on GCE node
I20260808 13:04:06.558491 11644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:06.559450 11644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:06.560678 11644 hybrid_clock.cc:648] HybridClock initialized: now 1786194246560610 us; error 64 us; skew 500 ppm
I20260808 13:04:06.561013 11644 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:06 dist-test-slave-t3q3 krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194246, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:06.564584 11644 init.cc:377] Logged in from keytab as oryx/127.1.124.131@KRBTEST.COM (short username oryx)
I20260808 13:04:06.566308 11644 webserver.cc:460] Webserver started at http://127.1.124.131:44991/ using document root <none> and password file <none>
I20260808 13:04:06.567070 11644 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:06.567230 11644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:06.567552 11644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:06.570825 11644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "068dd3cdc5dd4e8ca38c428773c5d616"
format_stamp: "Formatted at 2026-08-08 13:04:06 on dist-test-slave-t3q3"
server_key: "8f4f79ad94d98d0f7fd997b04612516c"
server_key_iv: "d26d096d9fad1f1b185092857911586c"
server_key_version: "encryptionkey@0"
I20260808 13:04:06.571674 11644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "068dd3cdc5dd4e8ca38c428773c5d616"
format_stamp: "Formatted at 2026-08-08 13:04:06 on dist-test-slave-t3q3"
server_key: "8f4f79ad94d98d0f7fd997b04612516c"
server_key_iv: "d26d096d9fad1f1b185092857911586c"
server_key_version: "encryptionkey@0"
I20260808 13:04:06.577625 11644 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 13:04:06.581719 11665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:06.583303 11644 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.005s sys 0.000s
I20260808 13:04:06.583519 11644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "068dd3cdc5dd4e8ca38c428773c5d616"
format_stamp: "Formatted at 2026-08-08 13:04:06 on dist-test-slave-t3q3"
server_key: "8f4f79ad94d98d0f7fd997b04612516c"
server_key_iv: "d26d096d9fad1f1b185092857911586c"
server_key_version: "encryptionkey@0"
I20260808 13:04:06.583779 11644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:06.586973 11434 heartbeater.cc:503] Master 127.1.124.190:39241 was elected leader, sending a full tablet report...
I20260808 13:04:06.611017 11644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:06.615720 11644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:06.616039 11644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:06.617148 11644 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:06.618600 11644 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:06.618716 11644 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:06.618957 11644 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:06.619065 11644 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:06.657467 11644 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:42219
I20260808 13:04:06.657518 11853 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:42219 every 8 connection(s)
I20260808 13:04:06.658802 11644 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:04:06.663357 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 11644
I20260808 13:04:06.663587 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:04:06.664045 1522 external_mini_cluster.cc:1428] Setting key a5655387bef3a72555f3bd9a6c387b46
Aug 08 13:04:06 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194246, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM
I20260808 13:04:06.679943 11857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:06.661378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:39881 (local address 127.1.124.190:39241)
0808 13:04:06.661902 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:06.661907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:06.662674 (+ 767us) server_negotiation.cc:407] Connection header received
0808 13:04:06.664878 (+ 2204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:06.664884 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:06.664980 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:06.665107 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:06.667260 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:06.668291 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:06.669895 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:06.670308 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:06.670421 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:06.673887 (+ 3466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:06.673891 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:06.673895 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:06.673927 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:06.676627 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:06.677337 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:06.677341 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:06.677343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:06.677462 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:06.677871 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:06.677875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:06.677878 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:06.678222 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:06.678336 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:06.679291 (+ 955us) server_negotiation.cc:299] Negotiation successful
0808 13:04:06.679516 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":288,"threads_started":1}
I20260808 13:04:06.681010 11861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:06.661749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.131:39881)
0808 13:04:06.662490 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:06.662536 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:06.664495 (+ 1959us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:06.665385 (+ 890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:06.665396 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:06.666130 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:06.666976 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:06.667003 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:06.668444 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:06.668448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:06.669735 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:06.669753 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:06.670508 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:06.670512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:06.670723 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:06.671366 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:06.671388 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:06.673679 (+ 2291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:06.676764 (+ 3085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:06.676772 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:06.676787 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:06.677228 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:06.677560 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:06.677563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:06.677565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:06.677755 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:06.678358 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:06.678367 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:06.678799 (+ 432us) client_negotiation.cc:769] Sending connection context
0808 13:04:06.679138 (+ 339us) client_negotiation.cc:240] Negotiation successful
0808 13:04:06.679658 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":311,"threads_started":1}
I20260808 13:04:06.682435 11855 heartbeater.cc:347] Connected to a master server at 127.1.124.190:39241
I20260808 13:04:06.682778 11855 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:06.683544 11855 heartbeater.cc:511] Master 127.1.124.190:39241 requested a full tablet report, sending...
I20260808 13:04:06.685078 11117 ts_manager.cc:194] Registered new tserver with Master: 068dd3cdc5dd4e8ca38c428773c5d616 (127.1.124.131:42219)
I20260808 13:04:06.685957 11117 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:39881
I20260808 13:04:06.693086 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.124.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.124.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.124.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.189.keytab.
Entry for principal oryx/127.1.124.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.189.keytab.
I20260808 13:04:06.755868 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.189:38679
--webserver_interface=127.1.124.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.189.keytab
--principal=oryx/127.1.124.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:41301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:39241,127.1.124.189:38679
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:07.107386 11864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:07.107659 11864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:07.107749 11864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:07.114981 11864 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:04:07.115101 11864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:07.115149 11864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:07.115211 11864 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:04:07.115254 11864 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:04:07.125074 11864 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:41301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:39241,127.1.124.189:38679
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.189.keytab
--principal=oryx/127.1.124.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.189:38679
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.124.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
W20260808 13:04:07.125401 11864 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.1.124.190:39241,127.1.124.189:38679'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 13:04:07.126484 11864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:07.127753 11864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 13:04:07.130722 11636 heartbeater.cc:503] Master 127.1.124.190:39241 was elected leader, sending a full tablet report...
W20260808 13:04:07.140730 11874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:07.140829 11872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:07.140789 11871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:07.141392 11864 server_base.cc:1034] running on GCE node
I20260808 13:04:07.142345 11864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:07.143857 11864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:07.145143 11864 hybrid_clock.cc:648] HybridClock initialized: now 1786194247145067 us; error 41 us; skew 500 ppm
I20260808 13:04:07.145488 11864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:07 dist-test-slave-t3q3 krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194247, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:07.148862 11864 init.cc:377] Logged in from keytab as oryx/127.1.124.189@KRBTEST.COM (short username oryx)
I20260808 13:04:07.150409 11864 webserver.cc:460] Webserver started at http://127.1.124.189:45579/ using document root <none> and password file <none>
I20260808 13:04:07.151129 11864 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:07.151263 11864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:07.151592 11864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:07.154640 11864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/data/instance:
uuid: "bd4cefb9655f4e4a920f42ad550c13c4"
format_stamp: "Formatted at 2026-08-08 13:04:07 on dist-test-slave-t3q3"
server_key: "0aad9b9e174ea0adfbc8157a03795f2e"
server_key_iv: "aa5d5234a7c44b3154e8af60bf6ce3d9"
server_key_version: "encryptionkey@0"
I20260808 13:04:07.155421 11864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/wal/instance:
uuid: "bd4cefb9655f4e4a920f42ad550c13c4"
format_stamp: "Formatted at 2026-08-08 13:04:07 on dist-test-slave-t3q3"
server_key: "0aad9b9e174ea0adfbc8157a03795f2e"
server_key_iv: "aa5d5234a7c44b3154e8af60bf6ce3d9"
server_key_version: "encryptionkey@0"
I20260808 13:04:07.160701 11864 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260808 13:04:07.164348 11889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:07.165658 11864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:04:07.165840 11864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/wal
uuid: "bd4cefb9655f4e4a920f42ad550c13c4"
format_stamp: "Formatted at 2026-08-08 13:04:07 on dist-test-slave-t3q3"
server_key: "0aad9b9e174ea0adfbc8157a03795f2e"
server_key_iv: "aa5d5234a7c44b3154e8af60bf6ce3d9"
server_key_version: "encryptionkey@0"
I20260808 13:04:07.166112 11864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:07.185523 11864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:07.189889 11864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:07.190179 11864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:07.209949 11864 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.189:38679
I20260808 13:04:07.209972 11975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.189:38679 every 8 connection(s)
I20260808 13:04:07.211387 11864 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/data/info.pb
I20260808 13:04:07.213375 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 11864
I20260808 13:04:07.216013 11976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:07.223196 11976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:04:07 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.124.189@KRBTEST.COM
Aug 08 13:04:07 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194247, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.189@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM
I20260808 13:04:07.237843 11979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:07.215834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41424 (local address 127.1.124.189:38679)
0808 13:04:07.216639 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:07.216651 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:07.216691 (+ 40us) server_negotiation.cc:407] Connection header received
0808 13:04:07.218332 (+ 1641us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:07.218355 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:07.218762 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:07.219204 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:07.220881 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.222225 (+ 1344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.223517 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.224335 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.224528 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:07.229675 (+ 5147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:07.229686 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:07.229700 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:07.229741 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:07.233455 (+ 3714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:07.234243 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:07.234253 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:07.234261 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:07.234493 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:07.234870 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:07.234878 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:07.234882 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:07.235508 (+ 626us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:07.235833 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:07.236259 (+ 426us) server_negotiation.cc:299] Negotiation successful
0808 13:04:07.236657 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":373,"threads_started":1}
I20260808 13:04:07.241899 11982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:07.225935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:57190)
0808 13:04:07.226647 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:07.226699 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:07.226844 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:07.227547 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:07.227555 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:07.228411 (+ 856us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:07.228772 (+ 361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:07.228786 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.230016 (+ 1230us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.230019 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:07.231578 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:07.231594 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.232622 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.232627 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:07.232799 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:07.233482 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:07.233509 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:07.236003 (+ 2494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:07.239326 (+ 3323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:07.239335 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:07.239339 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:07.239745 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:07.240155 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:07.240158 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:07.240160 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:07.240289 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:07.241059 (+ 770us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:07.241068 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:07.241233 (+ 165us) client_negotiation.cc:769] Sending connection context
0808 13:04:07.241387 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 13:04:07.241598 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":328,"threads_started":1}
I20260808 13:04:07.241907 11983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:07.225988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57190 (local address 127.1.124.190:39241)
0808 13:04:07.227231 (+ 1243us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:07.227236 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:07.227251 (+ 15us) server_negotiation.cc:407] Connection header received
0808 13:04:07.227301 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:07.227305 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:07.227360 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:07.227490 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:07.228949 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.229869 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.231731 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.232450 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.232581 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:07.236247 (+ 3666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:07.236254 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:07.236260 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:07.236317 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:07.239184 (+ 2867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:07.239867 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:07.239872 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:07.239875 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:07.240026 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:07.240408 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:07.240415 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:07.240419 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:07.240910 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:07.241028 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:07.241505 (+ 477us) server_negotiation.cc:299] Negotiation successful
0808 13:04:07.241671 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1137,"thread_start_us":1013,"threads_started":1}
I20260808 13:04:07.243639 11976 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:04:07 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194247, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.189@KRBTEST.COM for oryx/127.1.124.189@KRBTEST.COM
I20260808 13:04:07.255993 11982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:07.244456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:38679 (local address 127.0.0.1:41428)
0808 13:04:07.244666 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:07.244695 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:07.244778 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:07.245019 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:07.245022 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:07.245311 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:07.245436 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:07.245443 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.246353 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.246357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:07.247202 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:07.247212 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.247916 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.247920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:07.248198 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:07.248732 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:07.248747 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:07.250762 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:07.253908 (+ 3146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:07.253912 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:07.253914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:07.254267 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:07.254650 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:07.254653 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:07.254655 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:07.254789 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:07.255331 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:07.255334 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:07.255455 (+ 121us) client_negotiation.cc:769] Sending connection context
0808 13:04:07.255557 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 13:04:07.255763 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260808 13:04:07.256327 11979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:07.244442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41428 (local address 127.1.124.189:38679)
0808 13:04:07.244585 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:07.244589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:07.244760 (+ 171us) server_negotiation.cc:407] Connection header received
0808 13:04:07.244860 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:07.244864 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:07.244914 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:07.244986 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:07.245547 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.246228 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.247344 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.247782 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.247894 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:07.250948 (+ 3054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:07.250953 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:07.250957 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:07.251031 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:07.253760 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:07.254379 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:07.254383 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:07.254386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:07.254549 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:07.254914 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:07.254918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:07.254920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:07.255226 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:07.255345 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:07.255655 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 13:04:07.255905 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":53}
I20260808 13:04:07.260967 11976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4: Bootstrap starting.
I20260808 13:04:07.264112 11976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:07.265167 11976 log.cc:824] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4: Log is configured to *not* fsync() on all Append() calls
I20260808 13:04:07.267865 11976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4: No bootstrap required, opened a new log
I20260808 13:04:07.275406 11976 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } }
I20260808 13:04:07.275897 11976 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:07.276043 11976 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd4cefb9655f4e4a920f42ad550c13c4, State: Initialized, Role: FOLLOWER
I20260808 13:04:07.276762 11976 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } }
I20260808 13:04:07.277640 11985 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } } }
I20260808 13:04:07.277938 11985 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:04:07.278681 11976 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 13:04:07.289100 12002 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:04:07.289256 12002 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:04:07.292192 11864 master_runner.cc:428] Detected that this master bd4cefb9655f4e4a920f42ad550c13c4 is joining an existing cluster
I20260808 13:04:07.292312 11864 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.124.189:38679
I20260808 13:04:07.315733 11979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:07.299296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41432 (local address 127.1.124.189:38679)
0808 13:04:07.299507 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:07.299515 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:07.300090 (+ 575us) server_negotiation.cc:407] Connection header received
0808 13:04:07.300320 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:07.300325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:07.300387 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:07.300465 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:07.301937 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.303033 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.305064 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.305775 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.305871 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:07.308580 (+ 2709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:07.308586 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:07.308591 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:07.308628 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:07.312646 (+ 4018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:07.313539 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:07.313544 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:07.313548 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:07.313679 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:07.314090 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:07.314094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:07.314097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:07.314470 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:07.314629 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:07.315095 (+ 466us) server_negotiation.cc:299] Negotiation successful
0808 13:04:07.315344 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 13:04:07.316864 12012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:07.299280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:38679 (local address 127.0.0.1:41432)
0808 13:04:07.299968 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:07.299989 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:07.300195 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:07.300659 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:07.300664 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:07.301359 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:07.301791 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:07.301804 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.303254 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.303259 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:07.304906 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:07.304921 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.306186 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.306198 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:07.306359 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:07.307125 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:07.307152 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:07.308376 (+ 1224us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:07.312798 (+ 4422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:07.312803 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:07.312807 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:07.313410 (+ 603us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:07.313814 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:07.313818 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:07.313822 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:07.313982 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:07.314701 (+ 719us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:07.314706 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:07.314845 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 13:04:07.316000 (+ 1155us) client_negotiation.cc:240] Negotiation successful
0808 13:04:07.316344 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":389,"threads_started":1}
WARNING: no policy specified for oryx/127.1.124.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.124.188@KRBTEST.COM" created.
I20260808 13:04:07.327564 11983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:07.300289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57204 (local address 127.1.124.190:39241)
0808 13:04:07.300461 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:07.300466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:07.307577 (+ 7111us) server_negotiation.cc:407] Connection header received
0808 13:04:07.308080 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:07.308090 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:07.308198 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:07.308273 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:07.311347 (+ 3074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.312610 (+ 1263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.314938 (+ 2328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.315757 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.315874 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:07.320546 (+ 4672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:07.320554 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:07.320559 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:07.320604 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:07.324413 (+ 3809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:07.325094 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:07.325101 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:07.325106 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:07.325271 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:07.325805 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:07.325813 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:07.325817 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:07.326169 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:07.326386 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:07.326874 (+ 488us) server_negotiation.cc:299] Negotiation successful
0808 13:04:07.327155 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 13:04:07.327672 12014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:07.300717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:57204)
0808 13:04:07.307450 (+ 6733us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:07.307472 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:07.307891 (+ 419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:07.308511 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:07.308518 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:07.310620 (+ 2102us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:07.311146 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:07.311163 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.312826 (+ 1663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.312834 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:07.314768 (+ 1934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:07.314786 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.316602 (+ 1816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.316611 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:07.316839 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:07.317977 (+ 1138us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:07.318005 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:07.320293 (+ 2288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:07.324626 (+ 4333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:07.324635 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:07.324639 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:07.324987 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:07.325432 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:07.325438 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:07.325442 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:07.325613 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:07.326313 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:07.326318 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:07.326547 (+ 229us) client_negotiation.cc:769] Sending connection context
0808 13:04:07.326724 (+ 177us) client_negotiation.cc:240] Negotiation successful
0808 13:04:07.327104 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6567,"thread_start_us":6870,"threads_started":1}
Aug 08 13:04:07 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194244, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.190@KRBTEST.COM for oryx/127.1.124.189@KRBTEST.COM
I20260808 13:04:07.355161 12017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:07.335474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:38679 (local address 127.0.0.1:41446)
0808 13:04:07.336730 (+ 1256us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:07.336789 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:07.337105 (+ 316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:07.337646 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:07.337657 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:07.338434 (+ 777us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:07.338676 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:07.338686 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.339893 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.339899 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:07.341718 (+ 1819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:07.341737 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.342996 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.343003 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:07.343220 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:07.344128 (+ 908us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:07.344156 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:07.347422 (+ 3266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:07.351756 (+ 4334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:07.351775 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:07.351787 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:07.352458 (+ 671us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:07.352953 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:07.352975 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:07.352980 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:07.353182 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:07.353977 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:07.353987 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:07.354221 (+ 234us) client_negotiation.cc:769] Sending connection context
0808 13:04:07.354426 (+ 205us) client_negotiation.cc:240] Negotiation successful
0808 13:04:07.354838 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":966,"thread_start_us":810,"threads_started":1}
I20260808 13:04:07.355201 11979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:07.335535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41446 (local address 127.1.124.189:38679)
0808 13:04:07.335734 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:07.335742 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:07.336941 (+ 1199us) server_negotiation.cc:407] Connection header received
0808 13:04:07.337352 (+ 411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:07.337357 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:07.337455 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:07.337574 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:07.338860 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.339719 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.341876 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.342782 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.342951 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:07.347661 (+ 4710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:07.347668 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:07.347674 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:07.347722 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:07.351564 (+ 3842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:07.352621 (+ 1057us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:07.352627 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:07.352631 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:07.352807 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:07.353338 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:07.353343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:07.353346 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:07.353794 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:07.353947 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:07.354570 (+ 623us) server_negotiation.cc:299] Negotiation successful
0808 13:04:07.354829 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 13:04:07.357821 11117 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.124.189:38679: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4"
member_type: NON_VOTER
last_known_addr {
host: "127.1.124.189"
port: 38679
}
attrs {
promote: true
}
}
dest_uuid: "cdbf0bd6947747b9a2f3229974732e17"
cas_config_opid_index: -1
I20260808 13:04:07.359299 11117 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: NON_VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.124.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.188.keytab.
Entry for principal oryx/127.1.124.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.188.keytab.
I20260808 13:04:07.363066 12021 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bd4cefb9655f4e4a920f42ad550c13c4 (127.1.124.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: NON_VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: true } } }
I20260808 13:04:07.364769 11937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:07.364893 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.188:36769
--webserver_interface=127.1.124.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.188.keytab
--principal=oryx/127.1.124.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:41301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:39241,127.1.124.189:38679,127.1.124.188:36769
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:04:07.368427 11937 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 1 FOLLOWER]: Refusing update from remote peer cdbf0bd6947747b9a2f3229974732e17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 13:04:07.379142 12024 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: NON_VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:04:07.380465 12021 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.124.189:38679
I20260808 13:04:07.382602 12020 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cdbf0bd6947747b9a2f3229974732e17" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: NON_VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: true } } }
I20260808 13:04:07.382988 12020 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:07.383831 11864 master.cc:537] Master@127.1.124.189:38679 shutting down...
W20260808 13:04:07.386404 11081 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 13:04:07.389484 11081 consensus_peers.cc:487] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 -> Peer bd4cefb9655f4e4a920f42ad550c13c4 (127.1.124.189:38679): Couldn't send request to peer bd4cefb9655f4e4a920f42ad550c13c4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:04:07.397516 11864 raft_consensus.cc:2229] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:04:07.398034 11864 raft_consensus.cc:2258] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:04:07.398244 11864 tablet_replica.cc:330] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4: stopping tablet replica
I20260808 13:04:07.406488 11864 master.cc:552] Master@127.1.124.189:38679 shutdown complete.
I20260808 13:04:07.406749 11864 master_runner.cc:305] Clearing existing system tablet
I20260808 13:04:07.407485 11864 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:07.407716 11864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:07.411854 11864 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:04:07.414239 12030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:07.414655 11864 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:07.414932 11864 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 13:04:07.415019 11864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/wal
uuid: "bd4cefb9655f4e4a920f42ad550c13c4"
format_stamp: "Formatted at 2026-08-08 13:04:07 on dist-test-slave-t3q3"
server_key: "0aad9b9e174ea0adfbc8157a03795f2e"
server_key_iv: "aa5d5234a7c44b3154e8af60bf6ce3d9"
server_key_version: "encryptionkey@0"
I20260808 13:04:07.420423 11864 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:07.427860 11864 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 13:04:07.428100 11864 log.cc:1196] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 13:04:07.428586 11864 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4: Deleting consensus metadata
I20260808 13:04:07.428867 11864 master_runner.cc:315] Copying system tablet from 127.1.124.190:39241
I20260808 13:04:07.432118 11864 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4: tablet copy: Beginning tablet copy session from remote peer at address 127.1.124.190:39241
I20260808 13:04:07.450078 11983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:07.433477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57220 (local address 127.1.124.190:39241)
0808 13:04:07.433717 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:07.433724 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:07.433824 (+ 100us) server_negotiation.cc:407] Connection header received
0808 13:04:07.434120 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:07.434126 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:07.434235 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:07.434357 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:07.435495 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.436794 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.438690 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.439502 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.439650 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:07.442251 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:07.442261 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:07.442267 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:07.442320 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:07.446609 (+ 4289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:07.447486 (+ 877us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:07.447492 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:07.447495 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:07.447666 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:07.448150 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:07.448155 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:07.448159 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:07.448642 (+ 483us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:07.448842 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:07.449532 (+ 690us) server_negotiation.cc:299] Negotiation successful
0808 13:04:07.449778 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 13:04:07.451048 12039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:07.433121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:57220)
0808 13:04:07.433699 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:07.433721 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:07.433944 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:07.434409 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:07.434414 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:07.434996 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:07.435312 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:07.435326 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.436975 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.436981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:07.438503 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:07.438520 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.439675 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.439682 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:07.439884 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:07.440682 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:07.440707 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:07.441909 (+ 1202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:07.446782 (+ 4873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:07.446788 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:07.446793 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:07.447337 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:07.447800 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:07.447805 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:07.447809 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:07.447987 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:07.448793 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:07.448804 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:07.449005 (+ 201us) client_negotiation.cc:769] Sending connection context
0808 13:04:07.449192 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 13:04:07.450582 (+ 1390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":643,"threads_started":1}
I20260808 13:04:07.452557 11151 tablet_copy_service.cc:140] P cdbf0bd6947747b9a2f3229974732e17: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer bd4cefb9655f4e4a920f42ad550c13c4 ({username='oryx', principal='oryx/127.1.124.189@KRBTEST.COM'} at 127.0.0.1:57220)
I20260808 13:04:07.453192 11151 tablet_copy_service.cc:161] P cdbf0bd6947747b9a2f3229974732e17: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer bd4cefb9655f4e4a920f42ad550c13c4 at {username='oryx', principal='oryx/127.1.124.189@KRBTEST.COM'} at 127.0.0.1:57220: session id = bd4cefb9655f4e4a920f42ad550c13c4-00000000000000000000000000000000
I20260808 13:04:07.458810 11151 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 13:04:07.461881 11864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 13:04:07.469465 11864 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4: tablet copy: Starting download of 0 data blocks...
I20260808 13:04:07.469892 11864 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4: tablet copy: Starting download of 1 WAL segments...
I20260808 13:04:07.473933 11864 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 13:04:07.478662 11151 tablet_copy_service.cc:342] P cdbf0bd6947747b9a2f3229974732e17: Request end of tablet copy session bd4cefb9655f4e4a920f42ad550c13c4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.124.189@KRBTEST.COM'} at 127.0.0.1:57220
I20260808 13:04:07.479010 11151 tablet_copy_service.cc:433] P cdbf0bd6947747b9a2f3229974732e17: ending tablet copy session bd4cefb9655f4e4a920f42ad550c13c4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer bd4cefb9655f4e4a920f42ad550c13c4
I20260808 13:04:07.493099 11864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:07.495929 12040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:07.501684 12042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:07.509977 11864 server_base.cc:1034] running on GCE node
W20260808 13:04:07.510118 12045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:07.510715 11864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:07.511490 11864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:07.512715 11864 hybrid_clock.cc:648] HybridClock initialized: now 1786194247512659 us; error 31 us; skew 500 ppm
I20260808 13:04:07.512979 11864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:07 dist-test-slave-t3q3 krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194247, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:07.515463 11864 init.cc:377] Logged in from keytab as oryx/127.1.124.189@KRBTEST.COM (short username oryx)
I20260808 13:04:07.521080 11864 webserver.cc:460] Webserver started at http://127.1.124.189:41451/ using document root <none> and password file <none>
I20260808 13:04:07.521473 11864 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:07.521560 11864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:07.525038 11864 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 13:04:07.527405 12063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:07.527904 11864 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260808 13:04:07.528090 11864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/wal
uuid: "bd4cefb9655f4e4a920f42ad550c13c4"
format_stamp: "Formatted at 2026-08-08 13:04:07 on dist-test-slave-t3q3"
server_key: "0aad9b9e174ea0adfbc8157a03795f2e"
server_key_iv: "aa5d5234a7c44b3154e8af60bf6ce3d9"
server_key_version: "encryptionkey@0"
I20260808 13:04:07.528283 11864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:07.575129 11864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:07.579938 11864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:07.616942 11864 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.189:38679
I20260808 13:04:07.616966 12136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.189:38679 every 8 connection(s)
I20260808 13:04:07.618600 11864 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-1/data/info.pb
I20260808 13:04:07.624393 12140 sys_catalog.cc:263] Verifying existing consensus state
I20260808 13:04:07.628863 12140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4: Bootstrap starting.
I20260808 13:04:07.647320 12140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 13:04:07.647961 12140 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4: Bootstrap complete.
I20260808 13:04:07.649202 12140 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: NON_VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: true } }
I20260808 13:04:07.649495 12140 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: bd4cefb9655f4e4a920f42ad550c13c4, State: Initialized, Role: LEARNER
I20260808 13:04:07.649788 12140 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: NON_VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: true } }
I20260808 13:04:07.650753 12144 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: NON_VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: true } } }
I20260808 13:04:07.651052 12140 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:04:07.651243 12144 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [sys.catalog]: This master's current role is: LEARNER
I20260808 13:04:07.670089 12164 catalog_manager.cc:1259] Loaded cluster ID: 530b8a733f4e4536b5d2220088c480e1
I20260808 13:04:07.670248 12164 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4: loading cluster ID for follower catalog manager: success
I20260808 13:04:07.674796 12164 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4: acquiring CA information for follower catalog manager: success
I20260808 13:04:07.677785 12164 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 13:04:07.688863 11855 heartbeater.cc:503] Master 127.1.124.190:39241 was elected leader, sending a full tablet report...
I20260808 13:04:07.796799 12166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:07.789449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41456 (local address 127.1.124.189:38679)
0808 13:04:07.790045 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:07.790050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:07.790070 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:04:07.790133 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:07.790137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:07.790221 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:07.790323 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:07.791070 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.792203 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.794770 (+ 2567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.795629 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.795747 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:07.795935 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:07.796399 (+ 464us) server_negotiation.cc:299] Negotiation successful
0808 13:04:07.796508 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":483,"thread_start_us":332,"threads_started":1}
I20260808 13:04:07.796885 12017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:07.789246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:38679 (local address 127.0.0.1:41456)
0808 13:04:07.789427 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:07.789454 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:07.789562 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:07.790444 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:07.790452 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:07.790504 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:07.790793 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:07.790810 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.792375 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.792382 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:07.794569 (+ 2187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:07.794588 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.795846 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.795855 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:07.796176 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:07.796250 (+ 74us) client_negotiation.cc:769] Sending connection context
0808 13:04:07.796376 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 13:04:07.796555 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 13:04:07.798269 12105 raft_consensus.cc:1203] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260808 13:04:07.798941 12144 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdbf0bd6947747b9a2f3229974732e17. Latest consensus state: current_term: 1 leader_uuid: "cdbf0bd6947747b9a2f3229974732e17" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: NON_VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: true } } }
I20260808 13:04:07.799229 12144 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [sys.catalog]: This master's current role is: LEARNER
W20260808 13:04:07.827785 12025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:07.828197 12025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:07.828374 12025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:07.836274 12025 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:04:07.836386 12025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:07.836490 12025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:07.836577 12025 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:04:07.836665 12025 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:04:07.847186 12025 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:41301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:39241,127.1.124.189:38679,127.1.124.188:36769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.188.keytab
--principal=oryx/127.1.124.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.188:36769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.124.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:07.848846 12025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:07.850535 12025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:07.865888 12175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:07.865746 12180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:07.865918 12177 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:07.867709 12025 server_base.cc:1034] running on GCE node
I20260808 13:04:07.868762 12025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:07.870397 12025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:07.871762 12025 hybrid_clock.cc:648] HybridClock initialized: now 1786194247871622 us; error 62 us; skew 500 ppm
I20260808 13:04:07.872279 12025 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:07 dist-test-slave-t3q3 krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194247, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:07.876201 12025 init.cc:377] Logged in from keytab as oryx/127.1.124.188@KRBTEST.COM (short username oryx)
I20260808 13:04:07.877846 12025 webserver.cc:460] Webserver started at http://127.1.124.188:34343/ using document root <none> and password file <none>
I20260808 13:04:07.878543 12025 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:07.878682 12025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:07.878993 12025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:07.883385 12025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/data/instance:
uuid: "31b5517c6b78485b96efad3371700c56"
format_stamp: "Formatted at 2026-08-08 13:04:07 on dist-test-slave-t3q3"
server_key: "90ac739215e02ec1c472923d32658dfd"
server_key_iv: "cb8c69514f8263c8e8d5e89e5ff3178b"
server_key_version: "encryptionkey@0"
I20260808 13:04:07.884553 12025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/wal/instance:
uuid: "31b5517c6b78485b96efad3371700c56"
format_stamp: "Formatted at 2026-08-08 13:04:07 on dist-test-slave-t3q3"
server_key: "90ac739215e02ec1c472923d32658dfd"
server_key_iv: "cb8c69514f8263c8e8d5e89e5ff3178b"
server_key_version: "encryptionkey@0"
I20260808 13:04:07.891993 12025 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260808 13:04:07.896966 12188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:07.898782 12025 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260808 13:04:07.898981 12025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/wal
uuid: "31b5517c6b78485b96efad3371700c56"
format_stamp: "Formatted at 2026-08-08 13:04:07 on dist-test-slave-t3q3"
server_key: "90ac739215e02ec1c472923d32658dfd"
server_key_iv: "cb8c69514f8263c8e8d5e89e5ff3178b"
server_key_version: "encryptionkey@0"
I20260808 13:04:07.899221 12025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:07.917306 12025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:07.922403 12025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:07.922663 12025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:07.953274 12025 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.188:36769
I20260808 13:04:07.953281 12271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.188:36769 every 8 connection(s)
I20260808 13:04:07.954978 12025 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/data/info.pb
I20260808 13:04:07.959249 12272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:07.962169 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 12025
I20260808 13:04:07.970340 12272 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:04:07 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.124.188@KRBTEST.COM
Aug 08 13:04:07 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194247, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.188@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM
I20260808 13:04:07.989996 12278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:07.964889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34862 (local address 127.1.124.188:36769)
0808 13:04:07.965865 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:07.965878 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:07.965920 (+ 42us) server_negotiation.cc:407] Connection header received
0808 13:04:07.967764 (+ 1844us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:07.967788 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:07.968277 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:07.968721 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:07.970272 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.971659 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.973252 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.974699 (+ 1447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.974899 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:07.980958 (+ 6059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:07.980967 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:07.980979 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:07.981018 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:07.985084 (+ 4066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:07.985889 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:07.985898 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:07.985907 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:07.986076 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:07.986608 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:07.986613 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:07.986617 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:07.987271 (+ 654us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:07.987636 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:07.988219 (+ 583us) server_negotiation.cc:299] Negotiation successful
0808 13:04:07.988796 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":765,"thread_start_us":513,"threads_started":1}
I20260808 13:04:07.991693 12280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:07.973204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57224 (local address 127.1.124.190:39241)
0808 13:04:07.974205 (+ 1001us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:07.974211 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:07.974450 (+ 239us) server_negotiation.cc:407] Connection header received
0808 13:04:07.974634 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:07.974640 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:07.974722 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:07.974837 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:07.976535 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.977241 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.979555 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.980135 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.980288 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:07.984818 (+ 4530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:07.984824 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:07.984827 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:07.984861 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:07.988006 (+ 3145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:07.989120 (+ 1114us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:07.989126 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:07.989129 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:07.989269 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:07.989700 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:07.989707 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:07.989710 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:07.989991 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:07.990224 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:07.990775 (+ 551us) server_negotiation.cc:299] Negotiation successful
0808 13:04:07.991032 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":853,"thread_start_us":759,"threads_started":1}
I20260808 13:04:07.991725 12279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:07.973084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:57224)
0808 13:04:07.974293 (+ 1209us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:07.974344 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:07.974502 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:07.974919 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:07.974926 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:07.975778 (+ 852us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:07.976371 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:07.976386 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.977388 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.977407 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:07.979379 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:07.979400 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.980394 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.980401 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:07.980554 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:07.981284 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:07.981305 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:07.984607 (+ 3302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:07.988194 (+ 3587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:07.988206 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:07.988213 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:07.988966 (+ 753us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:07.989393 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:07.989398 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:07.989401 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:07.989582 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:07.990216 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:07.990224 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:07.990412 (+ 188us) client_negotiation.cc:769] Sending connection context
0808 13:04:07.990666 (+ 254us) client_negotiation.cc:240] Negotiation successful
0808 13:04:07.991107 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":954,"thread_start_us":627,"threads_started":1}
I20260808 13:04:07.993330 12272 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:04:07 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194247, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.188@KRBTEST.COM for oryx/127.1.124.189@KRBTEST.COM
I20260808 13:04:08.008179 12166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:07.994281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41466 (local address 127.1.124.189:38679)
0808 13:04:07.994494 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:07.994501 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:07.994613 (+ 112us) server_negotiation.cc:407] Connection header received
0808 13:04:07.994719 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:07.994722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:07.994783 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:07.994866 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:07.995712 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.996680 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.998201 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.998680 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.998783 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.002529 (+ 3746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:08.002536 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:08.002541 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:08.002581 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:08.005598 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.006376 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.006379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.006382 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.006520 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.006924 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.006928 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.006930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.007196 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:08.007308 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:08.007685 (+ 377us) server_negotiation.cc:299] Negotiation successful
0808 13:04:08.007864 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 13:04:08.008180 12279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:07.994293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:38679 (local address 127.0.0.1:41466)
0808 13:04:07.994528 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:07.994545 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:07.994637 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:07.994937 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:07.994940 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:07.995297 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:07.995563 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:07.995572 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.996839 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.996844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:07.998051 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:07.998064 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:07.998815 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:07.998820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:07.998952 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:07.999653 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:07.999672 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:08.002315 (+ 2643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:08.005746 (+ 3431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:08.005751 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:08.005753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:08.006264 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:08.006654 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:08.006658 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:08.006661 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:08.006825 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:08.007308 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:08.007312 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:08.007452 (+ 140us) client_negotiation.cc:769] Sending connection context
0808 13:04:08.007596 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 13:04:08.007840 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260808 13:04:08.009433 12272 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:04:08 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194247, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.188@KRBTEST.COM for oryx/127.1.124.188@KRBTEST.COM
I20260808 13:04:08.023489 12279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:08.010164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:36769 (local address 127.0.0.1:34876)
0808 13:04:08.010410 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:08.010430 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:08.010545 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:08.010855 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:08.010859 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:08.011186 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:08.011327 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:08.011334 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.012702 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.012706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:08.013532 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:08.013544 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.014246 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.014250 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:08.014414 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.015105 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:08.015124 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:08.017644 (+ 2520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:08.021223 (+ 3579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:08.021227 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:08.021229 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:08.021639 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:08.021981 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:08.021985 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:08.021987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:08.022136 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:08.022669 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:08.022673 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:08.022806 (+ 133us) client_negotiation.cc:769] Sending connection context
0808 13:04:08.022924 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 13:04:08.023097 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 13:04:08.023516 12278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:08.010210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34876 (local address 127.1.124.188:36769)
0808 13:04:08.010395 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:08.010399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:08.010533 (+ 134us) server_negotiation.cc:407] Connection header received
0808 13:04:08.010672 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:08.010675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:08.010734 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:08.010830 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:08.011483 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.012578 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.013667 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.014107 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.014216 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.017880 (+ 3664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:08.017885 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:08.017889 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:08.017926 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:08.021088 (+ 3162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.021751 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.021756 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.021759 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.021875 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.022251 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.022254 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.022257 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.022559 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:08.022680 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:08.023050 (+ 370us) server_negotiation.cc:299] Negotiation successful
0808 13:04:08.023231 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 13:04:08.029250 12272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56: Bootstrap starting.
I20260808 13:04:08.032985 12272 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:08.034456 12272 log.cc:824] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56: Log is configured to *not* fsync() on all Append() calls
I20260808 13:04:08.037670 12272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56: No bootstrap required, opened a new log
I20260808 13:04:08.046342 12272 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } }
I20260808 13:04:08.046743 12272 raft_consensus.cc:374] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:08.046881 12272 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 31b5517c6b78485b96efad3371700c56, State: Initialized, Role: FOLLOWER
I20260808 13:04:08.047535 12272 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } }
I20260808 13:04:08.048764 12282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } } }
I20260808 13:04:08.049125 12282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:04:08.049823 12272 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 13:04:08.062039 12301 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:04:08.062209 12301 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:04:08.067001 12025 master_runner.cc:428] Detected that this master 31b5517c6b78485b96efad3371700c56 is joining an existing cluster
I20260808 13:04:08.067206 12025 master_runner.cc:432] Initiating AddMaster RPC to add 127.1.124.188:36769
I20260808 13:04:08.094049 12166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:08.072837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41478 (local address 127.1.124.189:38679)
0808 13:04:08.073340 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:08.073351 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:08.074470 (+ 1119us) server_negotiation.cc:407] Connection header received
0808 13:04:08.074766 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:08.074772 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:08.074855 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:08.074933 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:08.077064 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.078495 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.080945 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.081776 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.081901 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.085254 (+ 3353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:08.085260 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:08.085264 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:08.085323 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:08.089995 (+ 4672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.091044 (+ 1049us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.091049 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.091053 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.091250 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.091926 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.091931 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.091934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.092347 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:08.092546 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:08.093368 (+ 822us) server_negotiation.cc:299] Negotiation successful
0808 13:04:08.093667 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311}
I20260808 13:04:08.095640 12308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:08.072937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:38679 (local address 127.0.0.1:41478)
0808 13:04:08.074354 (+ 1417us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:08.074375 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:08.074600 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:08.075159 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:08.075165 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:08.076251 (+ 1086us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:08.076874 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:08.076890 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.078815 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.078821 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:08.080771 (+ 1950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:08.080789 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.082327 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.082337 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:08.082539 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.083472 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:08.083497 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:08.085013 (+ 1516us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:08.090161 (+ 5148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:08.090166 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:08.090169 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:08.090871 (+ 702us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:08.091442 (+ 571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:08.091448 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:08.091452 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:08.091768 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:08.092645 (+ 877us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:08.092649 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:08.092903 (+ 254us) client_negotiation.cc:769] Sending connection context
0808 13:04:08.094356 (+ 1453us) client_negotiation.cc:240] Negotiation successful
0808 13:04:08.094734 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1196,"thread_start_us":905,"threads_started":1}
I20260808 13:04:08.097262 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 11219
I20260808 13:04:08.111159 12278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:08.075038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34890 (local address 127.1.124.188:36769)
0808 13:04:08.075240 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:08.075249 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:08.090307 (+ 15058us) server_negotiation.cc:407] Connection header received
0808 13:04:08.090711 (+ 404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:08.090717 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:08.090803 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:08.090915 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:08.092786 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.094104 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.099097 (+ 4993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.099856 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.099958 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.102628 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:08.102632 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:08.102636 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:08.102662 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:08.107243 (+ 4581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.108295 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.108301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.108306 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.108506 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.109120 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.109125 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.109128 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.109562 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:08.109711 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:08.110347 (+ 636us) server_negotiation.cc:299] Negotiation successful
0808 13:04:08.110715 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 13:04:08.112372 12311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:08.074541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:36769 (local address 127.0.0.1:34890)
0808 13:04:08.090167 (+ 15626us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:08.090188 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:08.090353 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:08.091125 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:08.091135 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:08.092115 (+ 980us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:08.092625 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:08.092637 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.094659 (+ 2022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.094666 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:08.098935 (+ 4269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:08.098955 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.100325 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.100332 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:08.100551 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.101247 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:08.101294 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:08.102419 (+ 1125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:08.107420 (+ 5001us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:08.107427 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:08.107433 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:08.108135 (+ 702us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:08.108699 (+ 564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:08.108708 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:08.108712 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:08.108996 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:08.109787 (+ 791us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:08.109792 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:08.110000 (+ 208us) client_negotiation.cc:769] Sending connection context
0808 13:04:08.111387 (+ 1387us) client_negotiation.cc:240] Negotiation successful
0808 13:04:08.111701 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15426,"thread_start_us":9754,"threads_started":1}
I20260808 13:04:08.114868 12280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:08.073067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57236 (local address 127.1.124.190:39241)
0808 13:04:08.073459 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:08.073465 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:08.084263 (+ 10798us) server_negotiation.cc:407] Connection header received
0808 13:04:08.084801 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:08.084811 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:08.084927 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:08.085015 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:08.087249 (+ 2234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.088931 (+ 1682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.093050 (+ 4119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.093730 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.093846 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.105820 (+ 11974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:08.105828 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:08.105832 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:08.105884 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:08.110483 (+ 4599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.111783 (+ 1300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.111789 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.111792 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.111960 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.112976 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.112982 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.112985 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.113399 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:08.113577 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:08.114435 (+ 858us) server_negotiation.cc:299] Negotiation successful
0808 13:04:08.114645 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253}
I20260808 13:04:08.115976 12312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:08.074133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:57236)
0808 13:04:08.084106 (+ 9973us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:08.084133 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:08.084586 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:08.085290 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:08.085299 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:08.086370 (+ 1071us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:08.087008 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:08.087027 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.089200 (+ 2173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.089210 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:08.092814 (+ 3604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:08.092830 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.094171 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.094179 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:08.094455 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.095254 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:08.095277 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:08.105469 (+ 10192us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:08.110715 (+ 5246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:08.110720 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:08.110723 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:08.111607 (+ 884us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:08.112131 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:08.112136 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:08.112139 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:08.112379 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:08.113669 (+ 1290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:08.113681 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:08.114099 (+ 418us) client_negotiation.cc:769] Sending connection context
0808 13:04:08.115168 (+ 1069us) client_negotiation.cc:240] Negotiation successful
0808 13:04:08.115497 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9746,"mutex_wait_us":29,"thread_start_us":15566,"threads_started":1}
Aug 08 13:04:08 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.124.129@KRBTEST.COM
Aug 08 13:04:08 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194244, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.190@KRBTEST.COM for oryx/127.1.124.188@KRBTEST.COM
I20260808 13:04:08.137328 12313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:08.099613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54130 (local address 127.1.124.129:35393)
0808 13:04:08.114485 (+ 14872us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:08.114495 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:08.114525 (+ 30us) server_negotiation.cc:407] Connection header received
0808 13:04:08.114597 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:08.114603 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:08.114825 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:08.115015 (+ 190us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:08.117124 (+ 2109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.118460 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.119876 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.120752 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.120897 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.127955 (+ 7058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:08.127972 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:08.127993 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:08.128150 (+ 157us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:08.132753 (+ 4603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.133725 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.134417 (+ 692us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.134424 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.134609 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.135100 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.135107 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.135111 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.135464 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:08.135802 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:08.136778 (+ 976us) server_negotiation.cc:299] Negotiation successful
0808 13:04:08.137034 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14679,"thread_start_us":14454,"threads_started":1}
I20260808 13:04:08.138932 11427 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:08.142983 12017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:08.123282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:36769 (local address 127.0.0.1:34896)
0808 13:04:08.123703 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:08.123726 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:08.123857 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:08.124322 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:08.124330 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:08.124952 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:08.125242 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:08.125256 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.126929 (+ 1673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.126934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:08.128385 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:08.128397 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.129491 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.129498 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:08.129681 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.130316 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:08.130344 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:08.133640 (+ 3296us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:08.139231 (+ 5591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:08.139237 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:08.139241 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:08.139793 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:08.140408 (+ 615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:08.140417 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:08.140424 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:08.140629 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:08.141471 (+ 842us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:08.141479 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:08.141772 (+ 293us) client_negotiation.cc:769] Sending connection context
0808 13:04:08.141953 (+ 181us) client_negotiation.cc:240] Negotiation successful
0808 13:04:08.142362 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231}
I20260808 13:04:08.144367 12278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:08.123323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34896 (local address 127.1.124.188:36769)
0808 13:04:08.123491 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:08.123496 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:08.123814 (+ 318us) server_negotiation.cc:407] Connection header received
0808 13:04:08.124020 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:08.124026 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:08.124160 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:08.124274 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:08.125437 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.126771 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.128538 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.129332 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.129474 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.133913 (+ 4439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:08.133921 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:08.133926 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:08.133965 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:08.139054 (+ 5089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.139969 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.140003 (+ 34us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.140006 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.140262 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.140826 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.140834 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.140840 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.141293 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:08.143587 (+ 2294us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:08.143721 (+ 134us) server_negotiation.cc:299] Negotiation successful
0808 13:04:08.143978 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 13:04:08.146193 11117 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.1.124.188:36769: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "31b5517c6b78485b96efad3371700c56"
member_type: NON_VOTER
last_known_addr {
host: "127.1.124.188"
port: 36769
}
attrs {
promote: true
}
}
dest_uuid: "cdbf0bd6947747b9a2f3229974732e17"
cas_config_opid_index: 5
I20260808 13:04:08.147332 11117 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: NON_VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: true } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: NON_VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: true } }
I20260808 13:04:08.149492 12024 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 31b5517c6b78485b96efad3371700c56 (127.1.124.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: NON_VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: true } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: NON_VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: true } } }
I20260808 13:04:08.153937 12226 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:08.156986 12105 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 1 LEARNER]: Refusing update from remote peer cdbf0bd6947747b9a2f3229974732e17: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 13:04:08.159713 12226 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 1 FOLLOWER]: Refusing update from remote peer cdbf0bd6947747b9a2f3229974732e17: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 13:04:08.162952 12316 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: NON_VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 13:04:08.174162 12318 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: NON_VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:04:08.175437 12199 tablet.cc:2369] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:04:08.177546 12105 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 31b5517c6b78485b96efad3371700c56 (127.1.124.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: NON_VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: true } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: NON_VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: true } } }
I20260808 13:04:08.182485 12144 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cdbf0bd6947747b9a2f3229974732e17" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: NON_VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: true } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: NON_VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: true } } }
I20260808 13:04:08.182790 12144 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [sys.catalog]: This master's current role is: LEARNER
I20260808 13:04:08.183431 12024 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.1.124.188:36769
I20260808 13:04:08.184310 12025 master.cc:537] Master@127.1.124.188:36769 shutting down...
I20260808 13:04:08.185316 12024 raft_consensus.cc:1050] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17: attempting to promote NON_VOTER bd4cefb9655f4e4a920f42ad550c13c4 to VOTER
I20260808 13:04:08.186470 12024 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: NON_VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: true } }
I20260808 13:04:08.191779 12316 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cdbf0bd6947747b9a2f3229974732e17" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: NON_VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: true } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: NON_VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: true } } } pending_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: NON_VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: true } } }
I20260808 13:04:08.192512 12316 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:08.193307 12105 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 1 LEARNER]: Refusing update from remote peer cdbf0bd6947747b9a2f3229974732e17: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 13:04:08.202558 12226 raft_consensus.cc:858] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: NON_VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: NON_VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: true } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: NON_VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: true } }
W20260808 13:04:08.203317 12226 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: NON_VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: NON_VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: true } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: NON_VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: true } }
I20260808 13:04:08.208892 12317 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 13:04:08.212844 12317 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bd4cefb9655f4e4a920f42ad550c13c4 (127.1.124.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: NON_VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: true } } }
W20260808 13:04:08.216418 11081 consensus_peers.cc:487] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 -> Peer 31b5517c6b78485b96efad3371700c56 (127.1.124.188:36769): Couldn't send request to peer 31b5517c6b78485b96efad3371700c56. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 13:04:08.217273 12316 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cdbf0bd6947747b9a2f3229974732e17" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: NON_VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: true } } }
I20260808 13:04:08.217723 12316 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:08.218039 12316 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cdbf0bd6947747b9a2f3229974732e17" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: NON_VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: true } } }
I20260808 13:04:08.218295 12316 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:08.218843 12105 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, bd4cefb9655f4e4a920f42ad550c13c4 (127.1.124.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: NON_VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: true } } }
I20260808 13:04:08.223202 12144 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cdbf0bd6947747b9a2f3229974732e17" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: NON_VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: true } } }
I20260808 13:04:08.223466 12144 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:04:08.236168 12226 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER bd4cefb9655f4e4a920f42ad550c13c4 (127.1.124.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: NON_VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: true } } }
I20260808 13:04:08.252441 12344 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cdbf0bd6947747b9a2f3229974732e17" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: NON_VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: true } } }
I20260808 13:04:08.252888 12344 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 13:04:08.253422 12282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdbf0bd6947747b9a2f3229974732e17. Latest consensus state: current_term: 1 leader_uuid: "cdbf0bd6947747b9a2f3229974732e17" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: NON_VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: true } } }
I20260808 13:04:08.253688 12282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 13:04:08.254094 12337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cdbf0bd6947747b9a2f3229974732e17" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: NON_VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: true } } }
I20260808 13:04:08.254361 12337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260808 13:04:08.256011 12025 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260808 13:04:08.256498 12025 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260808 13:04:08.256676 12025 tablet_replica.cc:330] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56: stopping tablet replica
I20260808 13:04:08.264564 12025 master.cc:552] Master@127.1.124.188:36769 shutdown complete.
I20260808 13:04:08.264657 12025 master_runner.cc:305] Clearing existing system tablet
I20260808 13:04:08.265097 12025 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:08.265218 12025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:08.268213 12025 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 13:04:08.269902 12346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:08.270288 12025 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:08.270542 12025 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260808 13:04:08.270640 12025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/wal
uuid: "31b5517c6b78485b96efad3371700c56"
format_stamp: "Formatted at 2026-08-08 13:04:07 on dist-test-slave-t3q3"
server_key: "90ac739215e02ec1c472923d32658dfd"
server_key_iv: "cb8c69514f8263c8e8d5e89e5ff3178b"
server_key_version: "encryptionkey@0"
I20260808 13:04:08.275417 12025 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:08.282290 12025 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 13:04:08.282449 12025 log.cc:1196] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 13:04:08.282929 12025 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56: Deleting consensus metadata
I20260808 13:04:08.283210 12025 master_runner.cc:315] Copying system tablet from 127.1.124.190:39241
I20260808 13:04:08.289769 12025 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56: tablet copy: Beginning tablet copy session from remote peer at address 127.1.124.190:39241
I20260808 13:04:08.306293 12280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:08.290724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57238 (local address 127.1.124.190:39241)
0808 13:04:08.290924 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:08.290930 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:08.291506 (+ 576us) server_negotiation.cc:407] Connection header received
0808 13:04:08.291722 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:08.291729 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:08.291814 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:08.291921 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:08.293047 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.294235 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.296199 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.296887 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.297023 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.299482 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:08.299488 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:08.299494 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:08.299534 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:08.303095 (+ 3561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.303927 (+ 832us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.303932 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.303936 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.304144 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.304607 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.304612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.304616 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.304980 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:08.305146 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:08.305693 (+ 547us) server_negotiation.cc:299] Negotiation successful
0808 13:04:08.305919 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 13:04:08.306375 12359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:08.290653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:57238)
0808 13:04:08.291357 (+ 704us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:08.291383 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:08.291552 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:08.291979 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:08.291984 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:08.292577 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:08.292836 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:08.292849 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.294396 (+ 1547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.294402 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:08.295998 (+ 1596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:08.296016 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.297064 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.297071 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:08.297262 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.298043 (+ 781us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:08.298068 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:08.299251 (+ 1183us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:08.303243 (+ 3992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:08.303250 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:08.303254 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:08.303785 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:08.304269 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:08.304276 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:08.304280 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:08.304473 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:08.305128 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:08.305138 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:08.305366 (+ 228us) client_negotiation.cc:769] Sending connection context
0808 13:04:08.305562 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 13:04:08.305905 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":570,"thread_start_us":430,"threads_started":1}
I20260808 13:04:08.307467 11151 tablet_copy_service.cc:140] P cdbf0bd6947747b9a2f3229974732e17: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 31b5517c6b78485b96efad3371700c56 ({username='oryx', principal='oryx/127.1.124.188@KRBTEST.COM'} at 127.0.0.1:57238)
I20260808 13:04:08.307650 11151 tablet_copy_service.cc:161] P cdbf0bd6947747b9a2f3229974732e17: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 31b5517c6b78485b96efad3371700c56 at {username='oryx', principal='oryx/127.1.124.188@KRBTEST.COM'} at 127.0.0.1:57238: session id = 31b5517c6b78485b96efad3371700c56-00000000000000000000000000000000
I20260808 13:04:08.311249 11151 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 13:04:08.313836 12025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 13:04:08.317718 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 11219
I20260808 13:04:08.320924 12025 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56: tablet copy: Starting download of 0 data blocks...
I20260808 13:04:08.321250 12025 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56: tablet copy: Starting download of 1 WAL segments...
I20260808 13:04:08.327895 12025 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 13:04:08.331560 11151 tablet_copy_service.cc:342] P cdbf0bd6947747b9a2f3229974732e17: Request end of tablet copy session 31b5517c6b78485b96efad3371700c56-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.1.124.188@KRBTEST.COM'} at 127.0.0.1:57238
I20260808 13:04:08.331717 11151 tablet_copy_service.cc:433] P cdbf0bd6947747b9a2f3229974732e17: ending tablet copy session 31b5517c6b78485b96efad3371700c56-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 31b5517c6b78485b96efad3371700c56
I20260808 13:04:08.342703 12025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:08.345827 12360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:08.349642 12362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:08.352401 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:35393
--local_ip_for_outbound_sockets=127.1.124.129
--tserver_master_addrs=127.1.124.190:39241
--webserver_port=42799
--webserver_interface=127.1.124.129
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.129.keytab
--principal=oryx/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:41301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.124.190:39241,127.1.124.189:38679
--tserver_master_addrs=127.1.124.190:39241,127.1.124.189:38679,127.1.124.188:36769 with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:08.356782 12366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:08.361505 12025 server_base.cc:1034] running on GCE node
I20260808 13:04:08.361881 12025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:08.362564 12025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:08.363772 12025 hybrid_clock.cc:648] HybridClock initialized: now 1786194248363730 us; error 39 us; skew 500 ppm
I20260808 13:04:08.364024 12025 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:08 dist-test-slave-t3q3 krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194248, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:08.366040 12025 init.cc:377] Logged in from keytab as oryx/127.1.124.188@KRBTEST.COM (short username oryx)
I20260808 13:04:08.367568 12025 webserver.cc:460] Webserver started at http://127.1.124.188:37029/ using document root <none> and password file <none>
I20260808 13:04:08.367964 12025 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:08.368134 12025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:08.370851 12025 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 13:04:08.372730 12376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:08.373167 12025 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260808 13:04:08.373304 12025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/wal
uuid: "31b5517c6b78485b96efad3371700c56"
format_stamp: "Formatted at 2026-08-08 13:04:07 on dist-test-slave-t3q3"
server_key: "90ac739215e02ec1c472923d32658dfd"
server_key_iv: "cb8c69514f8263c8e8d5e89e5ff3178b"
server_key_version: "encryptionkey@0"
I20260808 13:04:08.373504 12025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:08.388751 12025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:08.392316 12025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:08.417254 12025 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.188:36769
I20260808 13:04:08.417307 12465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.188:36769 every 8 connection(s)
I20260808 13:04:08.418306 12025 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/master-2/data/info.pb
I20260808 13:04:08.426046 12466 sys_catalog.cc:263] Verifying existing consensus state
I20260808 13:04:08.431505 12466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56: Bootstrap starting.
I20260808 13:04:08.445972 12466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 13:04:08.446524 12466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56: Bootstrap complete.
I20260808 13:04:08.447438 12466 raft_consensus.cc:348] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: NON_VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: true } }
I20260808 13:04:08.447715 12466 raft_consensus.cc:729] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 31b5517c6b78485b96efad3371700c56, State: Initialized, Role: LEARNER
I20260808 13:04:08.447966 12466 consensus_queue.cc:260] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: NON_VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: true } }
I20260808 13:04:08.451763 12470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: NON_VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: true } } }
I20260808 13:04:08.452170 12466 sys_catalog.cc:564] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:04:08.452209 12470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [sys.catalog]: This master's current role is: LEARNER
I20260808 13:04:08.472707 12497 catalog_manager.cc:1259] Loaded cluster ID: 530b8a733f4e4536b5d2220088c480e1
I20260808 13:04:08.472844 12497 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56: loading cluster ID for follower catalog manager: success
I20260808 13:04:08.475598 12497 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56: acquiring CA information for follower catalog manager: success
I20260808 13:04:08.477438 12497 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 13:04:08.672024 12498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:08.665063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:36769 (local address 127.0.0.1:34902)
0808 13:04:08.665631 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:08.665656 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:08.665831 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:08.666300 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:08.666306 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:08.666335 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:08.666608 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:08.666618 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.668254 (+ 1636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.668261 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:08.669548 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:08.669562 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.670888 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.670906 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:08.671155 (+ 249us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.671197 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 13:04:08.671312 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 13:04:08.671470 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":289,"threads_started":1}
I20260808 13:04:08.672729 12499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:08.665256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34902 (local address 127.1.124.188:36769)
0808 13:04:08.665954 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:08.665960 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:08.665981 (+ 21us) server_negotiation.cc:407] Connection header received
0808 13:04:08.666053 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:08.666056 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:08.666124 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:08.666280 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:08.666786 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.668034 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.669729 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.670613 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.670777 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.671149 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:08.671389 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 13:04:08.671582 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":391,"threads_started":1}
I20260808 13:04:08.673530 12419 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 13:04:08.674023 12470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cdbf0bd6947747b9a2f3229974732e17. Latest consensus state: current_term: 1 leader_uuid: "cdbf0bd6947747b9a2f3229974732e17" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: NON_VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: true } } }
I20260808 13:04:08.674369 12470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [sys.catalog]: This master's current role is: LEARNER
W20260808 13:04:08.769948 12369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:08.770270 12369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:08.770433 12369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:08.778249 12369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:08.778381 12369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:08.778578 12369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:04:08.789670 12369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:41301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.129.keytab
--principal=oryx/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:35393
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=42799
--tserver_master_addrs=127.1.124.190:39241,127.1.124.189:38679,127.1.124.188:36769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:08.791266 12369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:08.792723 12369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:08.806787 12508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:08.806839 12512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:08.806747 12507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:08.808218 12369 server_base.cc:1034] running on GCE node
I20260808 13:04:08.808897 12369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:08.809942 12369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:08.811259 12369 hybrid_clock.cc:648] HybridClock initialized: now 1786194248811174 us; error 57 us; skew 500 ppm
I20260808 13:04:08.811614 12369 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:08 dist-test-slave-t3q3 krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194248, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:08.815223 12369 init.cc:377] Logged in from keytab as oryx/127.1.124.129@KRBTEST.COM (short username oryx)
I20260808 13:04:08.817070 12369 webserver.cc:460] Webserver started at http://127.1.124.129:42799/ using document root <none> and password file <none>
I20260808 13:04:08.817819 12369 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:08.817989 12369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:08.823694 12369 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 13:04:08.826654 12523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:08.828325 12369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:04:08.828559 12369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "7a9c51158a034c87bc80b34266e50afe"
format_stamp: "Formatted at 2026-08-08 13:04:05 on dist-test-slave-t3q3"
server_key: "89c2a0266efd496f41f15be57e4d8c7b"
server_key_iv: "9e55978342bf49d8f59347bda5b529a4"
server_key_version: "encryptionkey@0"
I20260808 13:04:08.829777 12369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:08.860327 12369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:08.865257 12369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:08.865581 12369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:08.866760 12369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:08.868402 12369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:08.868531 12369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:08.868670 12369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:08.868790 12369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:08.909624 12369 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:35393
I20260808 13:04:08.909715 12717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:35393 every 8 connection(s)
I20260808 13:04:08.911207 12369 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:04:08.918349 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 12369
I20260808 13:04:08.918728 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 11443
Aug 08 13:04:08 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.124.130@KRBTEST.COM
Aug 08 13:04:08 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194248, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.189@KRBTEST.COM
Aug 08 13:04:08 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194248, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM
Aug 08 13:04:08 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194248, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.188@KRBTEST.COM
I20260808 13:04:08.959597 12738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:08.920252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40472 (local address 127.1.124.130:45593)
0808 13:04:08.931388 (+ 11136us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:08.931399 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:08.931435 (+ 36us) server_negotiation.cc:407] Connection header received
0808 13:04:08.931521 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:08.931531 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:08.931814 (+ 283us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:08.932011 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:08.934755 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.937449 (+ 2694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.941512 (+ 4063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.942336 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.942484 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.949207 (+ 6723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:08.949228 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:08.949249 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:08.949305 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:08.954483 (+ 5178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.956217 (+ 1734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.956230 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.956235 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.956737 (+ 502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.957260 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.957269 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.957275 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.957779 (+ 504us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:08.958092 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:08.958805 (+ 713us) server_negotiation.cc:299] Negotiation successful
0808 13:04:08.959098 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10936,"thread_start_us":10728,"threads_started":1}
I20260808 13:04:08.961009 11629 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:08.968326 12725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:08.918271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:51175 (local address 127.1.124.190:39241)
0808 13:04:08.931200 (+ 12929us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:08.931315 (+ 115us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:08.940893 (+ 9578us) server_negotiation.cc:407] Connection header received
0808 13:04:08.941188 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:08.941194 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:08.941287 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:08.941394 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:08.945203 (+ 3809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.946349 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.949412 (+ 3063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.950075 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.950222 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.956378 (+ 6156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:08.956385 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:08.956390 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:08.956433 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:08.961139 (+ 4706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.964675 (+ 3536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.964704 (+ 29us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.964711 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.965047 (+ 336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.966170 (+ 1123us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.966175 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.966178 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.966577 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:08.966743 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:08.967671 (+ 928us) server_negotiation.cc:299] Negotiation successful
0808 13:04:08.967924 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12793,"thread_start_us":12935,"threads_started":1}
I20260808 13:04:08.968652 12730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:08.934927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:38679 (local address 127.1.124.129:56649)
0808 13:04:08.941040 (+ 6113us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:08.941062 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:08.941178 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:08.942418 (+ 1240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:08.942425 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:08.943772 (+ 1347us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:08.944956 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:08.944986 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.947908 (+ 2922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.947913 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:08.950607 (+ 2694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:08.950619 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.951723 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.951731 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:08.951917 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.952531 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:08.952558 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:08.956134 (+ 3576us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:08.963219 (+ 7085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:08.963228 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:08.963234 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:08.964301 (+ 1067us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:08.964885 (+ 584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:08.964893 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:08.964899 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:08.965189 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:08.965954 (+ 765us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:08.965961 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:08.966433 (+ 472us) client_negotiation.cc:769] Sending connection context
0808 13:04:08.966740 (+ 307us) client_negotiation.cc:240] Negotiation successful
0808 13:04:08.967151 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5975,"mutex_wait_us":207,"spinlock_wait_cycles":17408,"thread_start_us":5365,"threads_started":1}
I20260808 13:04:08.969381 12499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:08.918682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:45009 (local address 127.1.124.188:36769)
0808 13:04:08.918907 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:08.918913 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:08.935982 (+ 17069us) server_negotiation.cc:407] Connection header received
0808 13:04:08.941391 (+ 5409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:08.941397 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:08.941479 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:08.941560 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:08.945183 (+ 3623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.946326 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.949441 (+ 3115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.950097 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.950201 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.958937 (+ 8736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:08.958946 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:08.958951 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:08.959002 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:08.962980 (+ 3978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.965895 (+ 2915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.965902 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.965906 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.966079 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.966555 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.966563 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.966569 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.966933 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:08.967086 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:08.968667 (+ 1581us) server_negotiation.cc:299] Negotiation successful
0808 13:04:08.968956 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260808 13:04:08.970304 12739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:08.931733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:56649 (local address 127.1.124.189:38679)
0808 13:04:08.941881 (+ 10148us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:08.941888 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:08.941912 (+ 24us) server_negotiation.cc:407] Connection header received
0808 13:04:08.941971 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:08.941976 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:08.942094 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:08.942210 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:08.946538 (+ 4328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.947735 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.950778 (+ 3043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.951459 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.951659 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.956389 (+ 4730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:08.956394 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:08.956398 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:08.956437 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:08.963060 (+ 6623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.964503 (+ 1443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.964511 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.964517 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.964729 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:08.965374 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:08.965382 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:08.965388 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:08.965812 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:08.965959 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:08.969863 (+ 3904us) server_negotiation.cc:299] Negotiation successful
0808 13:04:08.970033 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10027,"thread_start_us":9841,"threads_started":1}
I20260808 13:04:08.970607 12733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:08.919018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:36769 (local address 127.1.124.129:45009)
0808 13:04:08.935749 (+ 16731us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:08.935825 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:08.939478 (+ 3653us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:08.942412 (+ 2934us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:08.942425 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:08.944028 (+ 1603us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:08.944956 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:08.944987 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.946544 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.946548 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:08.949263 (+ 2715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:08.949278 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.950544 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.950550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:08.950809 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.953713 (+ 2904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:08.953738 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:08.958646 (+ 4908us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:08.963204 (+ 4558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:08.963214 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:08.963231 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:08.965766 (+ 2535us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:08.966179 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:08.966188 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:08.966194 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:08.966395 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:08.968157 (+ 1762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:08.968162 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:08.968334 (+ 172us) client_negotiation.cc:769] Sending connection context
0808 13:04:08.969772 (+ 1438us) client_negotiation.cc:240] Negotiation successful
0808 13:04:08.970121 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16326,"mutex_wait_us":326,"spinlock_wait_cycles":25600,"thread_start_us":11051,"threads_started":1}
I20260808 13:04:08.971097 12740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:08.919018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.129:51175)
0808 13:04:08.940746 (+ 21728us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:08.940778 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:08.940994 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:08.942766 (+ 1772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:08.942773 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:08.943894 (+ 1121us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:08.945020 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:08.945049 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.946533 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.946541 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:08.949259 (+ 2718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:08.949278 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:08.950379 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:08.950393 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:08.950816 (+ 423us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:08.951915 (+ 1099us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:08.951952 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:08.956162 (+ 4210us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:08.963217 (+ 7055us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:08.963221 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:08.963231 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:08.964465 (+ 1234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:08.965426 (+ 961us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:08.965436 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:08.965443 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:08.966019 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:08.967156 (+ 1137us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:08.967162 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:08.967324 (+ 162us) client_negotiation.cc:769] Sending connection context
0808 13:04:08.967490 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 13:04:08.967796 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21444,"mutex_wait_us":171,"spinlock_wait_cycles":29696,"thread_start_us":21592,"threads_started":1}
I20260808 13:04:08.971781 12723 heartbeater.cc:347] Connected to a master server at 127.1.124.189:38679
I20260808 13:04:08.971905 12718 heartbeater.cc:347] Connected to a master server at 127.1.124.188:36769
I20260808 13:04:08.972225 12723 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:08.972993 12723 heartbeater.cc:511] Master 127.1.124.189:38679 requested a full tablet report, sending...
I20260808 13:04:08.973117 12720 heartbeater.cc:347] Connected to a master server at 127.1.124.190:39241
I20260808 13:04:08.973277 12720 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:08.973875 12720 heartbeater.cc:511] Master 127.1.124.190:39241 requested a full tablet report, sending...
I20260808 13:04:08.972232 12718 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:08.974681 12718 heartbeater.cc:511] Master 127.1.124.188:36769 requested a full tablet report, sending...
I20260808 13:04:08.975334 11117 ts_manager.cc:194] Re-registered known tserver with Master: 7a9c51158a034c87bc80b34266e50afe (127.1.124.129:35393)
I20260808 13:04:08.976291 11117 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:51175
I20260808 13:04:08.976742 12400 ts_manager.cc:194] Registered new tserver with Master: 7a9c51158a034c87bc80b34266e50afe (127.1.124.129:35393)
I20260808 13:04:08.976737 12092 ts_manager.cc:194] Registered new tserver with Master: 7a9c51158a034c87bc80b34266e50afe (127.1.124.129:35393)
I20260808 13:04:09.055296 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 11443
I20260808 13:04:09.078099 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:45593
--local_ip_for_outbound_sockets=127.1.124.130
--tserver_master_addrs=127.1.124.190:39241
--webserver_port=33617
--webserver_interface=127.1.124.130
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.130.keytab
--principal=oryx/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:41301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.124.190:39241,127.1.124.189:38679
--tserver_master_addrs=127.1.124.190:39241,127.1.124.189:38679,127.1.124.188:36769 with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:04:09.090318 12317 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cdbf0bd6947747b9a2f3229974732e17" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: NON_VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: true } } }
I20260808 13:04:09.090762 12317 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:09.092650 12317 raft_consensus.cc:1050] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17: attempting to promote NON_VOTER 31b5517c6b78485b96efad3371700c56 to VOTER
I20260808 13:04:09.093268 12317 consensus_queue.cc:237] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: false } }
I20260808 13:04:09.095307 12105 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 1 FOLLOWER]: Refusing update from remote peer cdbf0bd6947747b9a2f3229974732e17: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260808 13:04:09.095369 12419 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 1 LEARNER]: Refusing update from remote peer cdbf0bd6947747b9a2f3229974732e17: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260808 13:04:09.095928 12748 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 13:04:09.096457 12747 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 13:04:09.099411 12748 raft_consensus.cc:2932] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 31b5517c6b78485b96efad3371700c56 (127.1.124.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: false } } }
I20260808 13:04:09.100284 12105 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 31b5517c6b78485b96efad3371700c56 (127.1.124.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: false } } }
I20260808 13:04:09.103379 12748 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "cdbf0bd6947747b9a2f3229974732e17" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: false } } }
I20260808 13:04:09.103655 12748 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:09.103834 12317 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cdbf0bd6947747b9a2f3229974732e17" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: false } } }
I20260808 13:04:09.104136 12317 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:09.104645 12317 sys_catalog.cc:455] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cdbf0bd6947747b9a2f3229974732e17" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: false } } }
I20260808 13:04:09.104910 12317 sys_catalog.cc:458] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17 [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:09.105073 12419 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 31b5517c6b78485b96efad3371700c56 (127.1.124.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: false } } }
I20260808 13:04:09.105749 12752 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cdbf0bd6947747b9a2f3229974732e17" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: false } } }
I20260808 13:04:09.106213 12752 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:04:09.108228 12470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cdbf0bd6947747b9a2f3229974732e17" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: false } } }
I20260808 13:04:09.108496 12470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [sys.catalog]: This master's current role is: FOLLOWER
W20260808 13:04:09.432350 12745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:09.432682 12745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:09.432850 12745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:09.440642 12745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:09.440819 12745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:09.441009 12745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:04:09.451112 12745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:41301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.130.keytab
--principal=oryx/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:45593
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=33617
--tserver_master_addrs=127.1.124.190:39241,127.1.124.189:38679,127.1.124.188:36769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:09.452695 12745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:09.454154 12745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:09.467878 12760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:09.467922 12761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:09.467988 12763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:09.469163 12745 server_base.cc:1034] running on GCE node
I20260808 13:04:09.469938 12745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:09.471091 12745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:09.472342 12745 hybrid_clock.cc:648] HybridClock initialized: now 1786194249472272 us; error 33 us; skew 500 ppm
I20260808 13:04:09.472760 12745 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:09 dist-test-slave-t3q3 krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194249, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:09.476527 12745 init.cc:377] Logged in from keytab as oryx/127.1.124.130@KRBTEST.COM (short username oryx)
I20260808 13:04:09.478041 12745 webserver.cc:460] Webserver started at http://127.1.124.130:33617/ using document root <none> and password file <none>
I20260808 13:04:09.478843 12745 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:09.479015 12745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:09.484927 12745 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 13:04:09.487895 12772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:09.489468 12745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 13:04:09.489674 12745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "001d8fb8a10e470b8ea6e80f680c6ebb"
format_stamp: "Formatted at 2026-08-08 13:04:06 on dist-test-slave-t3q3"
server_key: "97e4d5ba116a97b50d79f70c92966c7f"
server_key_iv: "46072441219b7f5315cf4b0dad59f0a3"
server_key_version: "encryptionkey@0"
I20260808 13:04:09.490694 12745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:09.516443 12745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:09.521112 12745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:09.521410 12745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:09.522452 12745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:09.523864 12745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:09.524004 12745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:09.524156 12745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:09.524253 12745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:09.574944 12745 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:45593
I20260808 13:04:09.574940 12960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:45593 every 8 connection(s)
I20260808 13:04:09.577050 12745 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:04:09.578287 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 12745
I20260808 13:04:09.578581 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 11644
Aug 08 13:04:09 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194249, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.188@KRBTEST.COM
Aug 08 13:04:09 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.124.131@KRBTEST.COM
I20260808 13:04:09.609649 12967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:09.581904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:54003 (local address 127.1.124.188:36769)
0808 13:04:09.585458 (+ 3554us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:09.585467 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:09.585492 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:04:09.585586 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:09.585594 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:09.585693 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:09.585834 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:09.589135 (+ 3301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:09.590469 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:09.592984 (+ 2515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:09.593641 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:09.593795 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:09.598745 (+ 4950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:09.598753 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:09.598758 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:09.598806 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:09.602864 (+ 4058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:09.603942 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:09.603950 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:09.603955 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:09.604160 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:09.604996 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:09.605002 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:09.605007 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:09.605424 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:09.605584 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:09.609034 (+ 3450us) server_negotiation.cc:299] Negotiation successful
0808 13:04:09.609293 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3395,"thread_start_us":3282,"threads_started":1}
I20260808 13:04:09.610419 12965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:09.580838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:36769 (local address 127.1.124.130:54003)
0808 13:04:09.581691 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:09.581748 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:09.584607 (+ 2859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:09.586733 (+ 2126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:09.586743 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:09.587673 (+ 930us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:09.588820 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:09.588853 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:09.590665 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:09.590671 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:09.592800 (+ 2129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:09.592816 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:09.594125 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:09.594135 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:09.594453 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:09.595211 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:09.595240 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:09.598442 (+ 3202us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:09.603040 (+ 4598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:09.603048 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:09.603067 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:09.603782 (+ 715us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:09.604320 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:09.604326 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:09.604331 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:09.604606 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:09.605672 (+ 1066us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:09.605693 (+ 21us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:09.606144 (+ 451us) client_negotiation.cc:769] Sending connection context
0808 13:04:09.606632 (+ 488us) client_negotiation.cc:240] Negotiation successful
0808 13:04:09.607237 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":3347,"threads_started":1}
I20260808 13:04:09.611856 12966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:09.581481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32966 (local address 127.1.124.131:42219)
0808 13:04:09.586527 (+ 5046us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:09.586537 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:09.586564 (+ 27us) server_negotiation.cc:407] Connection header received
0808 13:04:09.586628 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:09.586635 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:09.586873 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:09.589332 (+ 2459us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:09.589579 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:09.591042 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:09.592939 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:09.593815 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:09.594864 (+ 1049us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:09.603011 (+ 8147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:09.603027 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:09.603043 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:09.603089 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:09.607424 (+ 4335us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:09.608511 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:09.608519 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:09.608523 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:09.608691 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:09.609190 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:09.609198 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:09.609204 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:09.609617 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:09.610246 (+ 629us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:09.611150 (+ 904us) server_negotiation.cc:299] Negotiation successful
0808 13:04:09.611439 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4870,"thread_start_us":4733,"threads_started":1}
I20260808 13:04:09.612517 12962 heartbeater.cc:347] Connected to a master server at 127.1.124.188:36769
I20260808 13:04:09.612880 12962 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:09.613859 12962 heartbeater.cc:511] Master 127.1.124.188:36769 requested a full tablet report, sending...
I20260808 13:04:09.615144 11846 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:09.616617 12400 ts_manager.cc:194] Registered new tserver with Master: 001d8fb8a10e470b8ea6e80f680c6ebb (127.1.124.130:45593)
Aug 08 13:04:09 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194249, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.189@KRBTEST.COM
Aug 08 13:04:09 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194249, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM
I20260808 13:04:09.630288 12971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:09.589082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.130:36811)
0808 13:04:09.603815 (+ 14733us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:09.603836 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:09.604041 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:09.610188 (+ 6147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:09.610194 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:09.610869 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:09.611213 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:09.611226 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:09.613949 (+ 2723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:09.613954 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:09.615573 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:09.615589 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:09.617196 (+ 1607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:09.617202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:09.617396 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:09.618036 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:09.618070 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:09.621254 (+ 3184us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:09.626479 (+ 5225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:09.626498 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:09.626505 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:09.627220 (+ 715us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:09.627755 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:09.627763 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:09.627769 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:09.627947 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:09.628924 (+ 977us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:09.628932 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:09.629114 (+ 182us) client_negotiation.cc:769] Sending connection context
0808 13:04:09.629377 (+ 263us) client_negotiation.cc:240] Negotiation successful
0808 13:04:09.629749 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14568,"thread_start_us":15055,"threads_started":1}
I20260808 13:04:09.631176 12975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:09.601271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:45961 (local address 127.1.124.189:38679)
0808 13:04:09.605834 (+ 4563us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:09.605843 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:09.605865 (+ 22us) server_negotiation.cc:407] Connection header received
0808 13:04:09.605951 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:09.605959 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:09.606065 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:09.606188 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:09.608202 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:09.609640 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:09.613562 (+ 3922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:09.614200 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:09.614358 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:09.620353 (+ 5995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:09.620363 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:09.620371 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:09.620449 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:09.625742 (+ 5293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:09.627048 (+ 1306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:09.627056 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:09.627063 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:09.627265 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:09.627793 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:09.627801 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:09.627806 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:09.628277 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:09.628427 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:09.630122 (+ 1695us) server_negotiation.cc:299] Negotiation successful
0808 13:04:09.630468 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4413,"thread_start_us":6249,"threads_started":1}
I20260808 13:04:09.631892 12969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:09.589062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:36811 (local address 127.1.124.190:39241)
0808 13:04:09.597147 (+ 8085us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:09.597156 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:09.609887 (+ 12731us) server_negotiation.cc:407] Connection header received
0808 13:04:09.609960 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:09.609965 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:09.610041 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:09.610156 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:09.611442 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:09.612942 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:09.615744 (+ 2802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:09.616562 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:09.616720 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:09.621528 (+ 4808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:09.621538 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:09.621545 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:09.621595 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:09.626302 (+ 4707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:09.627394 (+ 1092us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:09.627402 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:09.627408 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:09.627607 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:09.628240 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:09.628248 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:09.628253 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:09.628677 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:09.628841 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:09.630879 (+ 2038us) server_negotiation.cc:299] Negotiation successful
0808 13:04:09.631144 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7943,"thread_start_us":7849,"threads_started":1}
I20260808 13:04:09.631767 12974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:09.601115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:38679 (local address 127.1.124.130:45961)
0808 13:04:09.604897 (+ 3782us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:09.604920 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:09.605054 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:09.606730 (+ 1676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:09.606738 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:09.607503 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:09.607893 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:09.607908 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:09.609828 (+ 1920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:09.609835 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:09.613383 (+ 3548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:09.613397 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:09.614416 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:09.614423 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:09.614651 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:09.615460 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:09.615490 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:09.620114 (+ 4624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:09.626022 (+ 5908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:09.626028 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:09.626035 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:09.626864 (+ 829us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:09.627427 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:09.627434 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:09.627440 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:09.627640 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:09.628519 (+ 879us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:09.628527 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:09.628748 (+ 221us) client_negotiation.cc:769] Sending connection context
0808 13:04:09.628975 (+ 227us) client_negotiation.cc:240] Negotiation successful
0808 13:04:09.629365 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3623,"spinlock_wait_cycles":22272,"thread_start_us":3610,"threads_started":1}
I20260808 13:04:09.633551 12963 heartbeater.cc:347] Connected to a master server at 127.1.124.190:39241
I20260808 13:04:09.633777 12963 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:09.634335 12963 heartbeater.cc:511] Master 127.1.124.190:39241 requested a full tablet report, sending...
I20260808 13:04:09.633494 12968 heartbeater.cc:347] Connected to a master server at 127.1.124.189:38679
I20260808 13:04:09.634951 12968 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:09.635442 12968 heartbeater.cc:511] Master 127.1.124.189:38679 requested a full tablet report, sending...
I20260808 13:04:09.635622 11117 ts_manager.cc:194] Re-registered known tserver with Master: 001d8fb8a10e470b8ea6e80f680c6ebb (127.1.124.130:45593)
I20260808 13:04:09.636956 11117 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:36811
I20260808 13:04:09.638181 12092 ts_manager.cc:194] Registered new tserver with Master: 001d8fb8a10e470b8ea6e80f680c6ebb (127.1.124.130:45593)
I20260808 13:04:09.716907 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 11644
I20260808 13:04:09.749483 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:42219
--local_ip_for_outbound_sockets=127.1.124.131
--tserver_master_addrs=127.1.124.190:39241
--webserver_port=44991
--webserver_interface=127.1.124.131
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.131.keytab
--principal=oryx/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:41301
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.1.124.190:39241,127.1.124.189:38679
--tserver_master_addrs=127.1.124.190:39241,127.1.124.189:38679,127.1.124.188:36769 with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:04:09.979837 12720 heartbeater.cc:503] Master 127.1.124.190:39241 was elected leader, sending a full tablet report...
W20260808 13:04:10.109441 12979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:10.109751 12979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:10.109848 12979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:10.117372 12979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:10.117504 12979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:10.117624 12979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:04:10.127810 12979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:41301
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.131.keytab
--principal=oryx/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:42219
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=44991
--tserver_master_addrs=127.1.124.190:39241,127.1.124.189:38679,127.1.124.188:36769
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:10.129362 12979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:10.130885 12979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:10.145433 12994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:10.145509 12987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:10.145465 12989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:10.146565 12979 server_base.cc:1034] running on GCE node
I20260808 13:04:10.147298 12979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:10.148492 12979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:10.149744 12979 hybrid_clock.cc:648] HybridClock initialized: now 1786194250149660 us; error 92 us; skew 500 ppm
I20260808 13:04:10.150099 12979 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:10 dist-test-slave-t3q3 krb5kdc[11037](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194250, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:10.153870 12979 init.cc:377] Logged in from keytab as oryx/127.1.124.131@KRBTEST.COM (short username oryx)
I20260808 13:04:10.155616 12979 webserver.cc:460] Webserver started at http://127.1.124.131:44991/ using document root <none> and password file <none>
I20260808 13:04:10.156452 12979 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:10.156625 12979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:10.162274 12979 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 13:04:10.165284 13003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:10.166924 12979 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:04:10.167245 12979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "068dd3cdc5dd4e8ca38c428773c5d616"
format_stamp: "Formatted at 2026-08-08 13:04:06 on dist-test-slave-t3q3"
server_key: "8f4f79ad94d98d0f7fd997b04612516c"
server_key_iv: "d26d096d9fad1f1b185092857911586c"
server_key_version: "encryptionkey@0"
I20260808 13:04:10.168507 12979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:10.184340 12979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:10.189198 12979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:10.189538 12979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:10.190670 12979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:10.192364 12979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:10.192513 12979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:10.192682 12979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:10.192790 12979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:10.229803 12979 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:42219
I20260808 13:04:10.229914 13192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:42219 every 8 connection(s)
I20260808 13:04:10.231516 12979 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:04:10.237670 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 12979
Aug 08 13:04:10 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194250, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM
Aug 08 13:04:10 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194250, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.188@KRBTEST.COM
Aug 08 13:04:10 dist-test-slave-t3q3 krb5kdc[11037](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194250, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.189@KRBTEST.COM
I20260808 13:04:10.291774 13204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.236477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:57683 (local address 127.1.124.190:39241)
0808 13:04:10.242893 (+ 6416us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:10.242898 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:10.262997 (+ 20099us) server_negotiation.cc:407] Connection header received
0808 13:04:10.265945 (+ 2948us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:10.265955 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:10.266076 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:10.266216 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:10.269223 (+ 3007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.270360 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.273084 (+ 2724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.274045 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.274327 (+ 282us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.280753 (+ 6426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:10.280763 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:10.280771 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:10.280827 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:10.286237 (+ 5410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:10.287335 (+ 1098us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:10.287345 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:10.287352 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:10.287584 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:10.288967 (+ 1383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:10.288976 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:10.288982 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:10.289479 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:10.289689 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:10.290719 (+ 1030us) server_negotiation.cc:299] Negotiation successful
0808 13:04:10.291169 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6306,"thread_start_us":6233,"threads_started":1}
I20260808 13:04:10.292783 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.236683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.131:57683)
0808 13:04:10.262828 (+ 26145us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:10.262849 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:10.265402 (+ 2553us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:10.266564 (+ 1162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:10.266577 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:10.267719 (+ 1142us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:10.268904 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:10.268939 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.270542 (+ 1603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.270547 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:10.272880 (+ 2333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:10.272897 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.274373 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.274380 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:10.274786 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.275660 (+ 874us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:10.275701 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:10.280435 (+ 4734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:10.286437 (+ 6002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:10.286447 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:10.286467 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:10.287156 (+ 689us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:10.288137 (+ 981us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:10.288144 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:10.288149 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:10.288401 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:10.289672 (+ 1271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:10.289684 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:10.290235 (+ 551us) client_negotiation.cc:769] Sending connection context
0808 13:04:10.290584 (+ 349us) client_negotiation.cc:240] Negotiation successful
0808 13:04:10.291202 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26001,"mutex_wait_us":53,"spinlock_wait_cycles":51968,"thread_start_us":26171,"threads_started":1}
I20260808 13:04:10.294782 13195 heartbeater.cc:347] Connected to a master server at 127.1.124.190:39241
I20260808 13:04:10.295156 13195 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:10.296180 13195 heartbeater.cc:511] Master 127.1.124.190:39241 requested a full tablet report, sending...
I20260808 13:04:10.298082 11117 ts_manager.cc:194] Re-registered known tserver with Master: 068dd3cdc5dd4e8ca38c428773c5d616 (127.1.124.131:42219)
I20260808 13:04:10.299237 11117 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:57683
I20260808 13:04:10.299615 13201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.235598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:36769 (local address 127.1.124.131:39851)
0808 13:04:10.261741 (+ 26143us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:10.261852 (+ 111us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:10.265402 (+ 3550us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:10.267766 (+ 2364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:10.267770 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:10.268461 (+ 691us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:10.268904 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:10.268939 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.272164 (+ 3225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.272170 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:10.273756 (+ 1586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:10.273776 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.275222 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.275231 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:10.275475 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.276598 (+ 1123us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:10.276622 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:10.288217 (+ 11595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:10.296591 (+ 8374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:10.296596 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:10.296600 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:10.297140 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:10.297654 (+ 514us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:10.297660 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:10.297663 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:10.297837 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:10.298574 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:10.298579 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:10.298746 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 13:04:10.298889 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 13:04:10.299182 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25623,"mutex_wait_us":3,"thread_start_us":494,"threads_started":1}
I20260808 13:04:10.300689 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.234969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:39851 (local address 127.1.124.188:36769)
0808 13:04:10.235564 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:10.235572 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:10.262081 (+ 26509us) server_negotiation.cc:407] Connection header received
0808 13:04:10.267340 (+ 5259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:10.267346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:10.267440 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:10.267535 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:10.270277 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.271396 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.274177 (+ 2781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.274818 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.274922 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.288482 (+ 13560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:10.288503 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:10.288507 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:10.288560 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:10.296379 (+ 7819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:10.297309 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:10.297318 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:10.297325 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:10.297508 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:10.297974 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:10.297979 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:10.297985 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:10.298348 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:10.298507 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:10.300032 (+ 1525us) server_negotiation.cc:299] Negotiation successful
0808 13:04:10.300336 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":283,"threads_started":1}
I20260808 13:04:10.312912 13210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.244328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:46547 (local address 127.1.124.189:38679)
0808 13:04:10.266674 (+ 22346us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:10.266681 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:10.266708 (+ 27us) server_negotiation.cc:407] Connection header received
0808 13:04:10.266776 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:10.266781 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:10.266863 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:10.267001 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:10.269324 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.270063 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.283173 (+ 13110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.283866 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.284698 (+ 832us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.288636 (+ 3938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:10.288644 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:10.288649 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:10.288695 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:10.303811 (+ 15116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:10.306200 (+ 2389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:10.306206 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:10.306209 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:10.306391 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:10.311143 (+ 4752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:10.311148 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:10.311151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:10.311541 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:10.311701 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:10.312268 (+ 567us) server_negotiation.cc:299] Negotiation successful
0808 13:04:10.312517 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22217,"thread_start_us":22788,"threads_started":1}
I20260808 13:04:10.312955 13193 heartbeater.cc:347] Connected to a master server at 127.1.124.188:36769
I20260808 13:04:10.313171 13193 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:10.313653 13193 heartbeater.cc:511] Master 127.1.124.188:36769 requested a full tablet report, sending...
I20260808 13:04:10.314079 13209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.244332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:38679 (local address 127.1.124.131:46547)
0808 13:04:10.264013 (+ 19681us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:10.264032 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:10.265419 (+ 1387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:10.267064 (+ 1645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:10.267072 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:10.268027 (+ 955us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:10.268974 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:10.268994 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.270549 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.270554 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:10.278870 (+ 8316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:10.278883 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.284143 (+ 5260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.284152 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:10.284358 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.284997 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:10.285025 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:10.287944 (+ 2919us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:10.303962 (+ 16018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:10.303968 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:10.303971 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:10.304571 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:10.306502 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:10.306510 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:10.306515 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:10.306785 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:10.311730 (+ 4945us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:10.311736 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:10.311933 (+ 197us) client_negotiation.cc:769] Sending connection context
0808 13:04:10.312158 (+ 225us) client_negotiation.cc:240] Negotiation successful
0808 13:04:10.312504 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19547,"mutex_wait_us":52,"spinlock_wait_cycles":768,"thread_start_us":18181,"threads_started":1}
I20260808 13:04:10.315196 12400 ts_manager.cc:194] Registered new tserver with Master: 068dd3cdc5dd4e8ca38c428773c5d616 (127.1.124.131:42219)
I20260808 13:04:10.315528 13196 heartbeater.cc:347] Connected to a master server at 127.1.124.189:38679
I20260808 13:04:10.315659 13196 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:10.316012 13196 heartbeater.cc:511] Master 127.1.124.189:38679 requested a full tablet report, sending...
I20260808 13:04:10.317308 12092 ts_manager.cc:194] Registered new tserver with Master: 068dd3cdc5dd4e8ca38c428773c5d616 (127.1.124.131:42219)
I20260808 13:04:10.322465 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.270939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37394 (local address 127.1.124.189:38679)
0808 13:04:10.278695 (+ 7756us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:10.278704 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:10.278729 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:04:10.278798 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:10.278804 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:10.278884 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:10.278987 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:10.284197 (+ 5210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.285477 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.304256 (+ 18779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.305029 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.306588 (+ 1559us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.310096 (+ 3508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:10.310102 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:10.310107 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:10.310147 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:10.318886 (+ 8739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:10.319925 (+ 1039us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:10.319931 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:10.319935 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:10.320146 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:10.320658 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:10.320662 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:10.320665 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:10.321035 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:10.321272 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:10.321764 (+ 492us) server_negotiation.cc:299] Negotiation successful
0808 13:04:10.322017 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7582,"thread_start_us":7434,"threads_started":1}
I20260808 13:04:10.326162 13218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.273285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33804 (local address 127.1.124.190:39241)
0808 13:04:10.283270 (+ 9985us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:10.283275 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:10.283296 (+ 21us) server_negotiation.cc:407] Connection header received
0808 13:04:10.283346 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:10.283351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:10.283416 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:10.283533 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:10.305174 (+ 21641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.305947 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.310058 (+ 4111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.310787 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.310926 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.318367 (+ 7441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:10.318376 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:10.318382 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:10.318429 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:10.322269 (+ 3840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:10.323136 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:10.323142 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:10.323145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:10.323288 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:10.323747 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:10.323754 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:10.323758 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:10.324203 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:10.324472 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:10.325567 (+ 1095us) server_negotiation.cc:299] Negotiation successful
0808 13:04:10.325863 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9832,"thread_start_us":9710,"threads_started":1}
I20260808 13:04:10.332221 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.282758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52228 (local address 127.1.124.188:36769)
0808 13:04:10.300927 (+ 18169us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:10.300932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:10.313806 (+ 12874us) server_negotiation.cc:407] Connection header received
0808 13:04:10.314100 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:10.314107 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:10.314209 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:10.314293 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:10.318051 (+ 3758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.318787 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.320429 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.321077 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.321194 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.324807 (+ 3613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:10.324818 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:10.324821 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:10.324851 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:10.328784 (+ 3933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:10.329736 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:10.329740 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:10.329742 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:10.329912 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:10.330394 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:10.330399 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:10.330401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:10.330831 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:10.331141 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:10.331509 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 13:04:10.331791 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18057,"thread_start_us":29104,"threads_started":1}
I20260808 13:04:10.331051 11117 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33804:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:04:10.339246 11117 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:04:10.373705 13239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.360532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:45593 (local address 127.0.0.1:55942)
0808 13:04:10.365228 (+ 4696us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:10.365253 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:10.365427 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:10.366039 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:10.366047 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:10.366080 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:10.366586 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:10.366601 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.367928 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.367936 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:10.370921 (+ 2985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:10.370944 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.372772 (+ 1828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.372779 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:10.373050 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.373086 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 13:04:10.373161 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 13:04:10.373341 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4532,"thread_start_us":7760,"threads_started":1}
I20260808 13:04:10.373863 13238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.360776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55942 (local address 127.1.124.130:45593)
0808 13:04:10.361502 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:10.361515 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:10.365414 (+ 3899us) server_negotiation.cc:407] Connection header received
0808 13:04:10.365580 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:10.365586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:10.365782 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:10.366027 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:10.366766 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.367782 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.371112 (+ 3330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.372591 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.372774 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.373078 (+ 304us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:10.373224 (+ 146us) server_negotiation.cc:299] Negotiation successful
0808 13:04:10.373438 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":1209,"threads_started":1}
I20260808 13:04:10.378368 12853 tablet_service.cc:1467] Processing CreateTablet for tablet 9044edf241da477a8eed34635a35c5dc (DEFAULT_TABLE table=test-table [id=96e7ed4d26b64b5783cf4ea99159eff0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:04:10.379012 13235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.360877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:35393 (local address 127.0.0.1:40038)
0808 13:04:10.368569 (+ 7692us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:10.368585 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:10.368834 (+ 249us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:10.369950 (+ 1116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:10.369957 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:10.369992 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:10.370553 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:10.370568 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.372686 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.374120 (+ 1434us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:10.376143 (+ 2023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:10.376162 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.377979 (+ 1817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.377985 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:10.378183 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.378218 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 13:04:10.378315 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 13:04:10.378460 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7562,"thread_start_us":4009,"threads_started":1}
I20260808 13:04:10.379132 13240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.361355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40038 (local address 127.1.124.129:35393)
0808 13:04:10.364656 (+ 3301us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:10.364669 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:10.368714 (+ 4045us) server_negotiation.cc:407] Connection header received
0808 13:04:10.369075 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:10.369083 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:10.369331 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:10.369554 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:10.370833 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.372523 (+ 1690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.376349 (+ 3826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.377726 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.377859 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.378063 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:10.378642 (+ 579us) server_negotiation.cc:299] Negotiation successful
0808 13:04:10.378797 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3081,"thread_start_us":2939,"threads_started":1}
I20260808 13:04:10.380049 12853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9044edf241da477a8eed34635a35c5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:10.382171 12610 tablet_service.cc:1467] Processing CreateTablet for tablet 9044edf241da477a8eed34635a35c5dc (DEFAULT_TABLE table=test-table [id=96e7ed4d26b64b5783cf4ea99159eff0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:04:10.383442 12610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9044edf241da477a8eed34635a35c5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:10.385443 13241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.368740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:42219 (local address 127.0.0.1:32970)
0808 13:04:10.372338 (+ 3598us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:10.372354 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:10.372461 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:10.377289 (+ 4828us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:10.377293 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:10.377321 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:10.377721 (+ 400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:10.377731 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.380516 (+ 2785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.380521 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:10.382491 (+ 1970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:10.382508 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.384611 (+ 2103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.384617 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:10.384810 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.384844 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 13:04:10.384949 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 13:04:10.385095 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3481,"thread_start_us":3295,"threads_started":1}
I20260808 13:04:10.385893 13244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.369164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32970 (local address 127.1.124.131:42219)
0808 13:04:10.376603 (+ 7439us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:10.376609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:10.376635 (+ 26us) server_negotiation.cc:407] Connection header received
0808 13:04:10.376764 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:10.376771 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:10.377000 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:10.377210 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:10.377905 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.380357 (+ 2452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.382664 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.384384 (+ 1720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.384539 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.384734 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:10.385027 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 13:04:10.385558 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7287,"thread_start_us":7109,"threads_started":1}
I20260808 13:04:10.389560 13094 tablet_service.cc:1467] Processing CreateTablet for tablet 9044edf241da477a8eed34635a35c5dc (DEFAULT_TABLE table=test-table [id=96e7ed4d26b64b5783cf4ea99159eff0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:04:10.390970 13094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9044edf241da477a8eed34635a35c5dc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:10.397637 13245 tablet_bootstrap.cc:492] T 9044edf241da477a8eed34635a35c5dc P 001d8fb8a10e470b8ea6e80f680c6ebb: Bootstrap starting.
I20260808 13:04:10.400839 13250 tablet_bootstrap.cc:492] T 9044edf241da477a8eed34635a35c5dc P 7a9c51158a034c87bc80b34266e50afe: Bootstrap starting.
I20260808 13:04:10.402546 13245 tablet_bootstrap.cc:654] T 9044edf241da477a8eed34635a35c5dc P 001d8fb8a10e470b8ea6e80f680c6ebb: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:10.403702 13245 log.cc:824] T 9044edf241da477a8eed34635a35c5dc P 001d8fb8a10e470b8ea6e80f680c6ebb: Log is configured to *not* fsync() on all Append() calls
I20260808 13:04:10.405043 13250 tablet_bootstrap.cc:654] T 9044edf241da477a8eed34635a35c5dc P 7a9c51158a034c87bc80b34266e50afe: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:10.406378 13253 tablet_bootstrap.cc:492] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616: Bootstrap starting.
I20260808 13:04:10.406862 13250 log.cc:824] T 9044edf241da477a8eed34635a35c5dc P 7a9c51158a034c87bc80b34266e50afe: Log is configured to *not* fsync() on all Append() calls
I20260808 13:04:10.407536 13245 tablet_bootstrap.cc:492] T 9044edf241da477a8eed34635a35c5dc P 001d8fb8a10e470b8ea6e80f680c6ebb: No bootstrap required, opened a new log
I20260808 13:04:10.407858 13245 ts_tablet_manager.cc:1397] T 9044edf241da477a8eed34635a35c5dc P 001d8fb8a10e470b8ea6e80f680c6ebb: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260808 13:04:10.410486 13253 tablet_bootstrap.cc:654] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:10.410538 13250 tablet_bootstrap.cc:492] T 9044edf241da477a8eed34635a35c5dc P 7a9c51158a034c87bc80b34266e50afe: No bootstrap required, opened a new log
I20260808 13:04:10.410818 13250 ts_tablet_manager.cc:1397] T 9044edf241da477a8eed34635a35c5dc P 7a9c51158a034c87bc80b34266e50afe: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260808 13:04:10.411855 13253 log.cc:824] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616: Log is configured to *not* fsync() on all Append() calls
I20260808 13:04:10.415288 13253 tablet_bootstrap.cc:492] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616: No bootstrap required, opened a new log
I20260808 13:04:10.415588 13253 ts_tablet_manager.cc:1397] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.004s
I20260808 13:04:10.416922 13245 raft_consensus.cc:348] T 9044edf241da477a8eed34635a35c5dc P 001d8fb8a10e470b8ea6e80f680c6ebb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 45593 } } peers { permanent_uuid: "7a9c51158a034c87bc80b34266e50afe" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35393 } } peers { permanent_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42219 } }
I20260808 13:04:10.417369 13245 raft_consensus.cc:374] T 9044edf241da477a8eed34635a35c5dc P 001d8fb8a10e470b8ea6e80f680c6ebb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:10.417479 13245 raft_consensus.cc:729] T 9044edf241da477a8eed34635a35c5dc P 001d8fb8a10e470b8ea6e80f680c6ebb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001d8fb8a10e470b8ea6e80f680c6ebb, State: Initialized, Role: FOLLOWER
I20260808 13:04:10.418112 13245 consensus_queue.cc:260] T 9044edf241da477a8eed34635a35c5dc P 001d8fb8a10e470b8ea6e80f680c6ebb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 45593 } } peers { permanent_uuid: "7a9c51158a034c87bc80b34266e50afe" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35393 } } peers { permanent_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42219 } }
I20260808 13:04:10.424588 13250 raft_consensus.cc:348] T 9044edf241da477a8eed34635a35c5dc P 7a9c51158a034c87bc80b34266e50afe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 45593 } } peers { permanent_uuid: "7a9c51158a034c87bc80b34266e50afe" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35393 } } peers { permanent_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42219 } }
I20260808 13:04:10.425040 13250 raft_consensus.cc:374] T 9044edf241da477a8eed34635a35c5dc P 7a9c51158a034c87bc80b34266e50afe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:10.425165 13250 raft_consensus.cc:729] T 9044edf241da477a8eed34635a35c5dc P 7a9c51158a034c87bc80b34266e50afe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a9c51158a034c87bc80b34266e50afe, State: Initialized, Role: FOLLOWER
I20260808 13:04:10.425864 13250 consensus_queue.cc:260] T 9044edf241da477a8eed34635a35c5dc P 7a9c51158a034c87bc80b34266e50afe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 45593 } } peers { permanent_uuid: "7a9c51158a034c87bc80b34266e50afe" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35393 } } peers { permanent_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42219 } }
I20260808 13:04:10.427706 13245 ts_tablet_manager.cc:1428] T 9044edf241da477a8eed34635a35c5dc P 001d8fb8a10e470b8ea6e80f680c6ebb: Time spent starting tablet: real 0.020s user 0.009s sys 0.012s
I20260808 13:04:10.427984 12963 heartbeater.cc:503] Master 127.1.124.190:39241 was elected leader, sending a full tablet report...
I20260808 13:04:10.429705 13253 raft_consensus.cc:348] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 45593 } } peers { permanent_uuid: "7a9c51158a034c87bc80b34266e50afe" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35393 } } peers { permanent_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42219 } }
I20260808 13:04:10.430331 13253 raft_consensus.cc:374] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:10.430474 13253 raft_consensus.cc:729] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068dd3cdc5dd4e8ca38c428773c5d616, State: Initialized, Role: FOLLOWER
I20260808 13:04:10.431174 13253 consensus_queue.cc:260] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 45593 } } peers { permanent_uuid: "7a9c51158a034c87bc80b34266e50afe" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35393 } } peers { permanent_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42219 } }
I20260808 13:04:10.434908 13250 ts_tablet_manager.cc:1428] T 9044edf241da477a8eed34635a35c5dc P 7a9c51158a034c87bc80b34266e50afe: Time spent starting tablet: real 0.024s user 0.015s sys 0.005s
I20260808 13:04:10.437661 13253 ts_tablet_manager.cc:1428] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616: Time spent starting tablet: real 0.022s user 0.016s sys 0.004s
I20260808 13:04:10.437973 13195 heartbeater.cc:503] Master 127.1.124.190:39241 was elected leader, sending a full tablet report...
W20260808 13:04:10.513309 13199 tablet.cc:2369] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:04:10.564659 13258 raft_consensus.cc:482] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:04:10.564972 13258 raft_consensus.cc:504] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 45593 } } peers { permanent_uuid: "7a9c51158a034c87bc80b34266e50afe" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35393 } } peers { permanent_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42219 } }
I20260808 13:04:10.566668 13258 leader_election.cc:288] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 001d8fb8a10e470b8ea6e80f680c6ebb (127.1.124.130:45593), 7a9c51158a034c87bc80b34266e50afe (127.1.124.129:35393)
I20260808 13:04:10.572928 13238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.567124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:54679 (local address 127.1.124.130:45593)
0808 13:04:10.567360 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:10.567366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:10.567636 (+ 270us) server_negotiation.cc:407] Connection header received
0808 13:04:10.567842 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:10.567849 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:10.567908 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:10.568012 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:10.568446 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.569171 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.570924 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.571784 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.571961 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.572037 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:10.572486 (+ 449us) server_negotiation.cc:299] Negotiation successful
0808 13:04:10.572664 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 13:04:10.572999 13209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.567068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:35393 (local address 127.1.124.131:34267)
0808 13:04:10.567473 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:10.567496 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:10.567628 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:10.567977 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:10.567981 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:10.568004 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:10.568267 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:10.568276 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.569332 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.569337 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:10.570777 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:10.570789 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.572023 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.572030 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:10.572286 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.572314 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 13:04:10.572447 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 13:04:10.572593 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":36,"spinlock_wait_cycles":3968}
I20260808 13:04:10.573240 13240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.567300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:34267 (local address 127.1.124.129:35393)
0808 13:04:10.567559 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:10.567565 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:10.567585 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:04:10.567793 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:10.567797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:10.567855 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:10.567956 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:10.568416 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.569181 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.570942 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.571825 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.572021 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.572123 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:10.572823 (+ 700us) server_negotiation.cc:299] Negotiation successful
0808 13:04:10.572976 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260808 13:04:10.573606 13201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.567256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:45593 (local address 127.1.124.131:54679)
0808 13:04:10.567502 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:10.567522 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:10.567685 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:10.568049 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:10.568054 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:10.568121 (+ 67us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:10.568302 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:10.568311 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.569308 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.569316 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:10.570759 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:10.570771 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.571962 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.571968 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:10.572273 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.572310 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 13:04:10.572412 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 13:04:10.572570 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":1,"spinlock_wait_cycles":896}
I20260808 13:04:10.574378 12643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9044edf241da477a8eed34635a35c5dc" candidate_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a9c51158a034c87bc80b34266e50afe" is_pre_election: true
I20260808 13:04:10.574990 12643 raft_consensus.cc:2454] T 9044edf241da477a8eed34635a35c5dc P 7a9c51158a034c87bc80b34266e50afe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068dd3cdc5dd4e8ca38c428773c5d616 in term 0.
I20260808 13:04:10.574971 12884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9044edf241da477a8eed34635a35c5dc" candidate_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" is_pre_election: true
I20260808 13:04:10.575587 12884 raft_consensus.cc:2454] T 9044edf241da477a8eed34635a35c5dc P 001d8fb8a10e470b8ea6e80f680c6ebb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 068dd3cdc5dd4e8ca38c428773c5d616 in term 0.
I20260808 13:04:10.575855 13007 leader_election.cc:302] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 068dd3cdc5dd4e8ca38c428773c5d616, 7a9c51158a034c87bc80b34266e50afe; no voters:
I20260808 13:04:10.576324 13258 raft_consensus.cc:2781] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:04:10.576547 13258 raft_consensus.cc:482] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:04:10.576696 13258 raft_consensus.cc:3037] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:10.578980 13258 raft_consensus.cc:504] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 45593 } } peers { permanent_uuid: "7a9c51158a034c87bc80b34266e50afe" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35393 } } peers { permanent_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42219 } }
I20260808 13:04:10.579627 13258 leader_election.cc:288] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616 [CANDIDATE]: Term 1 election: Requested vote from peers 001d8fb8a10e470b8ea6e80f680c6ebb (127.1.124.130:45593), 7a9c51158a034c87bc80b34266e50afe (127.1.124.129:35393)
I20260808 13:04:10.580134 12884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9044edf241da477a8eed34635a35c5dc" candidate_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb"
I20260808 13:04:10.580132 12643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9044edf241da477a8eed34635a35c5dc" candidate_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a9c51158a034c87bc80b34266e50afe"
I20260808 13:04:10.580335 12643 raft_consensus.cc:3037] T 9044edf241da477a8eed34635a35c5dc P 7a9c51158a034c87bc80b34266e50afe [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:10.580351 12884 raft_consensus.cc:3037] T 9044edf241da477a8eed34635a35c5dc P 001d8fb8a10e470b8ea6e80f680c6ebb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:10.582693 12884 raft_consensus.cc:2454] T 9044edf241da477a8eed34635a35c5dc P 001d8fb8a10e470b8ea6e80f680c6ebb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068dd3cdc5dd4e8ca38c428773c5d616 in term 1.
I20260808 13:04:10.583105 13009 leader_election.cc:302] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001d8fb8a10e470b8ea6e80f680c6ebb, 068dd3cdc5dd4e8ca38c428773c5d616; no voters:
I20260808 13:04:10.583410 13258 raft_consensus.cc:2781] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:04:10.583573 12643 raft_consensus.cc:2454] T 9044edf241da477a8eed34635a35c5dc P 7a9c51158a034c87bc80b34266e50afe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 068dd3cdc5dd4e8ca38c428773c5d616 in term 1.
I20260808 13:04:10.584050 13258 raft_consensus.cc:686] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616 [term 1 LEADER]: Becoming Leader. State: Replica: 068dd3cdc5dd4e8ca38c428773c5d616, State: Running, Role: LEADER
I20260808 13:04:10.584646 13258 consensus_queue.cc:237] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 45593 } } peers { permanent_uuid: "7a9c51158a034c87bc80b34266e50afe" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35393 } } peers { permanent_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42219 } }
I20260808 13:04:10.590355 11114 catalog_manager.cc:5495] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616 reported cstate change: term changed from 0 to 1, leader changed from <none> to 068dd3cdc5dd4e8ca38c428773c5d616 (127.1.124.131). New cstate: current_term: 1 leader_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 45593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a9c51158a034c87bc80b34266e50afe" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42219 } health_report { overall_health: HEALTHY } } }
W20260808 13:04:10.605232 12973 tablet.cc:2369] T 9044edf241da477a8eed34635a35c5dc P 001d8fb8a10e470b8ea6e80f680c6ebb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:04:10.633533 13244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.627452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32972 (local address 127.1.124.131:42219)
0808 13:04:10.627637 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:10.627641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:10.627729 (+ 88us) server_negotiation.cc:407] Connection header received
0808 13:04:10.627928 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:10.627932 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:10.628051 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:10.628255 (+ 204us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:04:10.628550 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.629447 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.631149 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.631710 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.631886 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.632386 (+ 500us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:04:10.632899 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:04:10.633127 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:10.633191 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 13:04:10.633258 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 13:04:10.642304 12884 raft_consensus.cc:1261] T 9044edf241da477a8eed34635a35c5dc P 001d8fb8a10e470b8ea6e80f680c6ebb [term 1 FOLLOWER]: Refusing update from remote peer 068dd3cdc5dd4e8ca38c428773c5d616: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:04:10.642295 12643 raft_consensus.cc:1261] T 9044edf241da477a8eed34635a35c5dc P 7a9c51158a034c87bc80b34266e50afe [term 1 FOLLOWER]: Refusing update from remote peer 068dd3cdc5dd4e8ca38c428773c5d616: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:04:10.643316 13260 consensus_queue.cc:1035] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a9c51158a034c87bc80b34266e50afe" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:04:10.643687 13258 consensus_queue.cc:1035] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616 [LEADER]: Connected to new peer: Peer: permanent_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 45593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:04:10.658052 13270 mvcc.cc:205] Tried to move back new op lower bound from 7316251650620329984 to 7316251650399113216. Current Snapshot: MvccSnapshot[applied={T|T < 7316251650620329984}]
I20260808 13:04:10.682919 11114 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:04:10.683240 11114 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33804:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:04:10.689788 11114 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cdbf0bd6947747b9a2f3229974732e17: Sending DeleteTablet for 3 replicas of tablet 9044edf241da477a8eed34635a35c5dc
I20260808 13:04:10.691406 13094 tablet_service.cc:1514] Processing DeleteTablet for tablet 9044edf241da477a8eed34635a35c5dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:04:10 UTC) from {username='oryx', principal='oryx/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:32970
I20260808 13:04:10.691530 12853 tablet_service.cc:1514] Processing DeleteTablet for tablet 9044edf241da477a8eed34635a35c5dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:04:10 UTC) from {username='oryx', principal='oryx/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:55942
I20260808 13:04:10.691545 12610 tablet_service.cc:1514] Processing DeleteTablet for tablet 9044edf241da477a8eed34635a35c5dc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:04:10 UTC) from {username='oryx', principal='oryx/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:40038
I20260808 13:04:10.703734 13282 tablet_replica.cc:330] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616: stopping tablet replica
I20260808 13:04:10.704375 13282 raft_consensus.cc:2229] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:04:10.704643 13283 tablet_replica.cc:330] T 9044edf241da477a8eed34635a35c5dc P 7a9c51158a034c87bc80b34266e50afe: stopping tablet replica
I20260808 13:04:10.705006 13283 raft_consensus.cc:2229] T 9044edf241da477a8eed34635a35c5dc P 7a9c51158a034c87bc80b34266e50afe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:04:10.705181 13282 raft_consensus.cc:2258] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:04:10.705540 13283 raft_consensus.cc:2258] T 9044edf241da477a8eed34635a35c5dc P 7a9c51158a034c87bc80b34266e50afe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:04:10.706166 13281 tablet_replica.cc:330] T 9044edf241da477a8eed34635a35c5dc P 001d8fb8a10e470b8ea6e80f680c6ebb: stopping tablet replica
I20260808 13:04:10.706686 13281 raft_consensus.cc:2229] T 9044edf241da477a8eed34635a35c5dc P 001d8fb8a10e470b8ea6e80f680c6ebb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:04:10.707244 13281 raft_consensus.cc:2258] T 9044edf241da477a8eed34635a35c5dc P 001d8fb8a10e470b8ea6e80f680c6ebb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:04:10.707438 13283 ts_tablet_manager.cc:1905] T 9044edf241da477a8eed34635a35c5dc P 7a9c51158a034c87bc80b34266e50afe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:10.709611 13281 ts_tablet_manager.cc:1905] T 9044edf241da477a8eed34635a35c5dc P 001d8fb8a10e470b8ea6e80f680c6ebb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:10.710304 13282 ts_tablet_manager.cc:1905] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:10.711220 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 11055
I20260808 13:04:10.712502 11168 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:10.715474 13283 ts_tablet_manager.cc:1918] T 9044edf241da477a8eed34635a35c5dc P 7a9c51158a034c87bc80b34266e50afe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:04:10.715653 13283 log.cc:1196] T 9044edf241da477a8eed34635a35c5dc P 7a9c51158a034c87bc80b34266e50afe: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/9044edf241da477a8eed34635a35c5dc
I20260808 13:04:10.716280 13283 ts_tablet_manager.cc:1939] T 9044edf241da477a8eed34635a35c5dc P 7a9c51158a034c87bc80b34266e50afe: Deleting consensus metadata
I20260808 13:04:10.718398 13281 ts_tablet_manager.cc:1918] T 9044edf241da477a8eed34635a35c5dc P 001d8fb8a10e470b8ea6e80f680c6ebb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:04:10.718659 13281 log.cc:1196] T 9044edf241da477a8eed34635a35c5dc P 001d8fb8a10e470b8ea6e80f680c6ebb: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/9044edf241da477a8eed34635a35c5dc
I20260808 13:04:10.718961 13282 ts_tablet_manager.cc:1918] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:04:10.719199 13282 log.cc:1196] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/9044edf241da477a8eed34635a35c5dc
I20260808 13:04:10.719467 13281 ts_tablet_manager.cc:1939] T 9044edf241da477a8eed34635a35c5dc P 001d8fb8a10e470b8ea6e80f680c6ebb: Deleting consensus metadata
I20260808 13:04:10.719923 13282 ts_tablet_manager.cc:1939] T 9044edf241da477a8eed34635a35c5dc P 068dd3cdc5dd4e8ca38c428773c5d616: Deleting consensus metadata
I20260808 13:04:10.747287 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.694626 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37400 (local address 127.1.124.189:38679)
0808 13:04:10.694832 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:10.694841 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:10.709648 (+ 14807us) server_negotiation.cc:407] Connection header received
0808 13:04:10.709890 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:10.709896 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:10.709985 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:10.710088 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:10.723239 (+ 13151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.724771 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.727201 (+ 2430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.727854 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.728009 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.741435 (+ 13426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:10.741441 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:10.741444 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:10.741494 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:10.744391 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:10.745162 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:10.745168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:10.745176 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:10.745305 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:10.745737 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:10.745754 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:10.745757 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:10.746054 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:10.746349 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:10.746686 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 13:04:10.746921 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 13:04:10.762542 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.710554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52242 (local address 127.1.124.188:36769)
0808 13:04:10.710751 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:10.710757 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:10.747507 (+ 36750us) server_negotiation.cc:407] Connection header received
0808 13:04:10.747684 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:10.747692 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:10.747791 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:10.747912 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:10.749984 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.751051 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.752742 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.753305 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.753501 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.756537 (+ 3036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:10.756542 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:10.756545 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:10.756581 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:10.759498 (+ 2917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:10.760258 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:10.760263 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:10.760268 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:10.760415 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:10.760892 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:10.760901 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:10.760903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:10.761246 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:10.761510 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:10.761860 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 13:04:10.762089 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 13:04:10.919252 11089 catalog_manager.cc:4841] TS 001d8fb8a10e470b8ea6e80f680c6ebb (127.1.124.130:45593): tablet 9044edf241da477a8eed34635a35c5dc (table test-table [id=96e7ed4d26b64b5783cf4ea99159eff0]) successfully deleted
W20260808 13:04:10.919551 11089 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.1.124.130:45593, user_credentials={real_user=slave}} blocked reactor thread for 198102us
I20260808 13:04:10.920251 11089 catalog_manager.cc:4841] TS 068dd3cdc5dd4e8ca38c428773c5d616 (127.1.124.131:42219): tablet 9044edf241da477a8eed34635a35c5dc (table test-table [id=96e7ed4d26b64b5783cf4ea99159eff0]) successfully deleted
I20260808 13:04:10.920866 11087 catalog_manager.cc:4841] TS 7a9c51158a034c87bc80b34266e50afe (127.1.124.129:35393): tablet 9044edf241da477a8eed34635a35c5dc (table test-table [id=96e7ed4d26b64b5783cf4ea99159eff0]) successfully deleted
W20260808 13:04:10.921016 11087 outbound_call.cc:322] RPC callback for RPC call kudu.tserver.TabletServerAdminService.DeleteTablet -> {remote=127.1.124.129:35393, user_credentials={real_user=slave}} blocked reactor thread for 202712us
I20260808 13:04:10.921545 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 11055
I20260808 13:04:10.987537 13215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.968298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37416 (local address 127.1.124.189:38679)
0808 13:04:10.968590 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:10.968600 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:10.969737 (+ 1137us) server_negotiation.cc:407] Connection header received
0808 13:04:10.969920 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:10.969929 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:10.970042 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:10.970167 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:10.972939 (+ 2772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.974104 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.975691 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.976370 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.976502 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.980825 (+ 4323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:10.980833 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:10.980838 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:10.980881 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:10.984453 (+ 3572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:10.985336 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:10.985342 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:10.985346 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:10.985497 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:10.985908 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:10.985913 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:10.985916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:10.986276 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:10.986507 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:10.986968 (+ 461us) server_negotiation.cc:299] Negotiation successful
0808 13:04:10.987190 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260808 13:04:10.989369 13200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:10.970009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52246 (local address 127.1.124.188:36769)
0808 13:04:10.970215 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:10.970223 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:10.970613 (+ 390us) server_negotiation.cc:407] Connection header received
0808 13:04:10.970911 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:10.970916 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:10.971004 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:10.971114 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:10.974118 (+ 3004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.975293 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.977098 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:10.977819 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:10.977960 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:10.981398 (+ 3438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:10.981404 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:10.981407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:10.981444 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:10.984889 (+ 3445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:10.986582 (+ 1693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:10.986588 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:10.986591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:10.986745 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:10.987778 (+ 1033us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:10.987783 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:10.987787 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:10.988156 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:10.988394 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:10.988840 (+ 446us) server_negotiation.cc:299] Negotiation successful
0808 13:04:10.989058 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 13:04:11.666992 13296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:11.666243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.129:60259)
0808 13:04:11.666689 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:11.666811 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":206,"threads_started":1}
W20260808 13:04:11.667433 12720 heartbeater.cc:644] Failed to heartbeat to 127.1.124.190:39241 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
I20260808 13:04:11.676414 13297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:11.675066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.131:35899)
0808 13:04:11.675700 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:11.676174 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":510,"thread_start_us":369,"threads_started":1}
W20260808 13:04:11.676913 13195 heartbeater.cc:644] Failed to heartbeat to 127.1.124.190:39241 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
I20260808 13:04:11.677397 13298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:11.676362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.130:51827)
0808 13:04:11.677075 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:11.677224 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":589,"thread_start_us":450,"threads_started":1}
W20260808 13:04:11.677780 12963 heartbeater.cc:644] Failed to heartbeat to 127.1.124.190:39241 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
I20260808 13:04:12.190510 13300 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cdbf0bd6947747b9a2f3229974732e17)
I20260808 13:04:12.190894 13301 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader cdbf0bd6947747b9a2f3229974732e17)
I20260808 13:04:12.190908 13300 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: false } }
I20260808 13:04:12.191244 13301 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: false } }
I20260808 13:04:12.192469 13300 leader_election.cc:288] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cdbf0bd6947747b9a2f3229974732e17 (127.1.124.190:39241), bd4cefb9655f4e4a920f42ad550c13c4 (127.1.124.189:38679)
I20260808 13:04:12.194213 13304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:12.192766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:34196)
0808 13:04:12.193567 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:12.193871 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":302,"threads_started":1}
W20260808 13:04:12.194520 12068 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
I20260808 13:04:12.192636 13301 leader_election.cc:288] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers cdbf0bd6947747b9a2f3229974732e17 (127.1.124.190:39241), 31b5517c6b78485b96efad3371700c56 (127.1.124.188:36769)
I20260808 13:04:12.204713 13303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:12.200274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:34184)
0808 13:04:12.204043 (+ 3769us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:12.204328 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3641,"thread_start_us":2586,"threads_started":1}
I20260808 13:04:12.205106 13312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:12.202454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:34200)
0808 13:04:12.204636 (+ 2182us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:12.204834 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2024,"thread_start_us":1239,"threads_started":1}
W20260808 13:04:12.205084 12382 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
W20260808 13:04:12.205394 12068 leader_election.cc:334] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cdbf0bd6947747b9a2f3229974732e17 (127.1.124.190:39241): Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
I20260808 13:04:12.206828 13303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:12.206398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:34206)
0808 13:04:12.206543 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:12.206621 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 13:04:12.208254 12382 leader_election.cc:334] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer cdbf0bd6947747b9a2f3229974732e17 (127.1.124.190:39241): Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
I20260808 13:04:12.210361 13304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:12.199723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:36769 (local address 127.0.0.1:52252)
0808 13:04:12.200731 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:12.200761 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:12.200966 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:12.202943 (+ 1977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:12.202951 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:12.202991 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:12.204238 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:12.204253 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:12.206848 (+ 2595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:12.206855 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:12.208301 (+ 1446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:12.208318 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:12.209423 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:12.209430 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:12.209659 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:12.209769 (+ 110us) client_negotiation.cc:769] Sending connection context
0808 13:04:12.209907 (+ 138us) client_negotiation.cc:240] Negotiation successful
0808 13:04:12.210049 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"mutex_wait_us":23}
I20260808 13:04:12.210407 13305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:12.192924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37432 (local address 127.1.124.189:38679)
0808 13:04:12.203975 (+ 11051us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:12.203979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:12.203993 (+ 14us) server_negotiation.cc:407] Connection header received
0808 13:04:12.204037 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:12.204040 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:12.204157 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:12.204286 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:12.205343 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:12.206562 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:12.208563 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:12.209530 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:12.209659 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:12.209861 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:12.209949 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 13:04:12.210071 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":10946,"thread_start_us":10346,"threads_started":1}
I20260808 13:04:12.210533 13310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:12.192710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:38679 (local address 127.0.0.1:37432)
0808 13:04:12.202585 (+ 9875us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:12.202609 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:12.202766 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:12.204339 (+ 1573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:12.204344 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:12.204371 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:12.205145 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:12.205161 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:12.206704 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:12.206709 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:12.208256 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:12.208291 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:12.209694 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:12.209699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:12.209807 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:12.209827 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 13:04:12.209933 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 13:04:12.210067 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9679,"thread_start_us":11084,"threads_started":1}
I20260808 13:04:12.211004 13306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:12.193856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52252 (local address 127.1.124.188:36769)
0808 13:04:12.202263 (+ 8407us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:12.202275 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:12.202303 (+ 28us) server_negotiation.cc:407] Connection header received
0808 13:04:12.202393 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:12.202399 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:12.202479 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:12.202609 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:12.204454 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:12.206693 (+ 2239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:12.208489 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:12.209256 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:12.209362 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:12.209427 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:12.209964 (+ 537us) server_negotiation.cc:299] Negotiation successful
0808 13:04:12.210088 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":8230,"thread_start_us":8089,"threads_started":1}
I20260808 13:04:12.211450 12105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "31b5517c6b78485b96efad3371700c56" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" is_pre_election: true
I20260808 13:04:12.211585 12419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "31b5517c6b78485b96efad3371700c56" is_pre_election: true
I20260808 13:04:12.211884 12419 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4cefb9655f4e4a920f42ad550c13c4 in term 1.
I20260808 13:04:12.211870 12105 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 31b5517c6b78485b96efad3371700c56 in term 1.
I20260808 13:04:12.212529 12065 leader_election.cc:302] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31b5517c6b78485b96efad3371700c56, bd4cefb9655f4e4a920f42ad550c13c4; no voters: cdbf0bd6947747b9a2f3229974732e17
I20260808 13:04:12.212576 12377 leader_election.cc:302] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31b5517c6b78485b96efad3371700c56, bd4cefb9655f4e4a920f42ad550c13c4; no voters: cdbf0bd6947747b9a2f3229974732e17
I20260808 13:04:12.213002 13301 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 13:04:12.213002 13300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 13:04:12.213146 13301 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cdbf0bd6947747b9a2f3229974732e17)
I20260808 13:04:12.213148 13300 raft_consensus.cc:482] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 1 FOLLOWER]: Starting leader election (detected failure of leader cdbf0bd6947747b9a2f3229974732e17)
I20260808 13:04:12.213243 13301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:04:12.213279 13300 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:04:12.216475 13300 raft_consensus.cc:504] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: false } }
I20260808 13:04:12.216475 13301 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: false } }
I20260808 13:04:12.217244 13301 leader_election.cc:288] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [CANDIDATE]: Term 2 election: Requested vote from peers cdbf0bd6947747b9a2f3229974732e17 (127.1.124.190:39241), 31b5517c6b78485b96efad3371700c56 (127.1.124.188:36769)
I20260808 13:04:12.217255 13300 leader_election.cc:288] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [CANDIDATE]: Term 2 election: Requested vote from peers cdbf0bd6947747b9a2f3229974732e17 (127.1.124.190:39241), bd4cefb9655f4e4a920f42ad550c13c4 (127.1.124.189:38679)
I20260808 13:04:12.217942 13310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:12.217444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:34210)
0808 13:04:12.217638 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:12.217788 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 13:04:12.217905 12105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "31b5517c6b78485b96efad3371700c56" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "bd4cefb9655f4e4a920f42ad550c13c4"
I20260808 13:04:12.217918 12419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "31b5517c6b78485b96efad3371700c56"
I20260808 13:04:12.218303 12105 raft_consensus.cc:2379] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 31b5517c6b78485b96efad3371700c56 in current term 2: Already voted for candidate bd4cefb9655f4e4a920f42ad550c13c4 in this term.
I20260808 13:04:12.220636 13310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:12.218612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:34230)
0808 13:04:12.218757 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:12.220509 (+ 1752us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 13:04:12.220722 12419 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate bd4cefb9655f4e4a920f42ad550c13c4 in current term 2: Already voted for candidate 31b5517c6b78485b96efad3371700c56 in this term.
W20260808 13:04:12.220875 12382 leader_election.cc:334] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cdbf0bd6947747b9a2f3229974732e17 (127.1.124.190:39241): Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
I20260808 13:04:12.222572 13304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:12.222164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:34208)
0808 13:04:12.222316 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:12.222389 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 13:04:12.223073 12377 leader_election.cc:302] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 31b5517c6b78485b96efad3371700c56; no voters: bd4cefb9655f4e4a920f42ad550c13c4, cdbf0bd6947747b9a2f3229974732e17
I20260808 13:04:12.223407 13300 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 13:04:12.223809 13304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:12.223372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:34248)
0808 13:04:12.223583 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:12.223675 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260808 13:04:12.224020 12068 leader_election.cc:334] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer cdbf0bd6947747b9a2f3229974732e17 (127.1.124.190:39241): Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
I20260808 13:04:12.224189 12068 leader_election.cc:302] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd4cefb9655f4e4a920f42ad550c13c4; no voters: 31b5517c6b78485b96efad3371700c56, cdbf0bd6947747b9a2f3229974732e17
I20260808 13:04:12.224431 13301 raft_consensus.cc:2726] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 13:04:12.668993 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:12.668306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.129:36379)
0808 13:04:12.668736 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:12.668822 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":208,"threads_started":1}
I20260808 13:04:12.679299 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:12.677733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.131:57533)
0808 13:04:12.678862 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:12.678993 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":997,"thread_start_us":881,"threads_started":1}
I20260808 13:04:12.679774 13317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:12.678556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.130:36123)
0808 13:04:12.679318 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:12.679417 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":531,"threads_started":1}
I20260808 13:04:13.670933 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:13.670102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.129:59925)
0808 13:04:13.670649 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:13.670749 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":301,"threads_started":1}
I20260808 13:04:13.681382 13322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:13.680272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.131:59349)
0808 13:04:13.681032 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:13.681142 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":644,"thread_start_us":503,"threads_started":1}
I20260808 13:04:13.682359 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:13.680883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.130:45513)
0808 13:04:13.682141 (+ 1258us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:13.682222 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1131,"thread_start_us":1017,"threads_started":1}
I20260808 13:04:13.766937 13324 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:04:13.767212 13324 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: false } }
I20260808 13:04:13.767930 13324 leader_election.cc:288] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers cdbf0bd6947747b9a2f3229974732e17 (127.1.124.190:39241), 31b5517c6b78485b96efad3371700c56 (127.1.124.188:36769)
I20260808 13:04:13.768601 12419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "31b5517c6b78485b96efad3371700c56" is_pre_election: true
I20260808 13:04:13.768926 12419 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd4cefb9655f4e4a920f42ad550c13c4 in term 2.
I20260808 13:04:13.769122 13325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:13.768254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:34486)
0808 13:04:13.768743 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:13.768883 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":248,"threads_started":1}
I20260808 13:04:13.769451 12065 leader_election.cc:302] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 31b5517c6b78485b96efad3371700c56, bd4cefb9655f4e4a920f42ad550c13c4; no voters:
I20260808 13:04:13.769766 13324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260808 13:04:13.769901 13324 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:04:13.770025 13324 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 2 FOLLOWER]: Advancing to term 3
I20260808 13:04:13.770623 13325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:13.770217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:34500)
0808 13:04:13.770392 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:13.770488 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 13:04:13.770841 12068 leader_election.cc:334] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer cdbf0bd6947747b9a2f3229974732e17 (127.1.124.190:39241): Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
I20260808 13:04:13.772569 13324 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: false } }
I20260808 13:04:13.773166 13324 leader_election.cc:288] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [CANDIDATE]: Term 3 election: Requested vote from peers cdbf0bd6947747b9a2f3229974732e17 (127.1.124.190:39241), 31b5517c6b78485b96efad3371700c56 (127.1.124.188:36769)
I20260808 13:04:13.773780 13325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:13.773362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:34504)
0808 13:04:13.773538 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:13.773639 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 13:04:13.773751 12419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "31b5517c6b78485b96efad3371700c56"
I20260808 13:04:13.773994 12419 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 2 FOLLOWER]: Advancing to term 3
I20260808 13:04:13.774812 13325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:13.774423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:34512)
0808 13:04:13.774569 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:13.774643 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 13:04:13.774960 12068 leader_election.cc:334] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer cdbf0bd6947747b9a2f3229974732e17 (127.1.124.190:39241): Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
I20260808 13:04:13.776465 12419 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd4cefb9655f4e4a920f42ad550c13c4 in term 3.
I20260808 13:04:13.776930 12065 leader_election.cc:302] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 31b5517c6b78485b96efad3371700c56, bd4cefb9655f4e4a920f42ad550c13c4; no voters: cdbf0bd6947747b9a2f3229974732e17
I20260808 13:04:13.777237 13324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 13:04:13.777652 13324 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [term 3 LEADER]: Becoming Leader. State: Replica: bd4cefb9655f4e4a920f42ad550c13c4, State: Running, Role: LEADER
I20260808 13:04:13.778035 13324 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: false } }
I20260808 13:04:13.780050 13327 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd4cefb9655f4e4a920f42ad550c13c4. Latest consensus state: current_term: 3 leader_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: false } } }
I20260808 13:04:13.780441 13327 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:13.781104 13333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:04:13.785032 13333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:04:13.785872 13333 catalog_manager.cc:1259] Loaded cluster ID: 530b8a733f4e4536b5d2220088c480e1
I20260808 13:04:13.785944 13333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:04:13.787111 13333 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:04:13.787946 13333 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4: Loaded TSK: 0
I20260808 13:04:13.788875 13333 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:04:13.803539 12090 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37416:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:04:13.810683 12090 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:04:13.815282 13325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:13.814738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:34522)
0808 13:04:13.814945 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:13.815077 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260808 13:04:13.815461 12419 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [term 3 FOLLOWER]: Refusing update from remote peer bd4cefb9655f4e4a920f42ad550c13c4: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260808 13:04:13.816424 13325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:13.815981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:34536)
0808 13:04:13.816213 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:13.816308 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 13:04:13.816293 13327 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260808 13:04:13.816679 12068 consensus_peers.cc:487] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 -> Peer cdbf0bd6947747b9a2f3229974732e17 (127.1.124.190:39241): Couldn't send request to peer cdbf0bd6947747b9a2f3229974732e17. Status: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 13:04:13.819072 13337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd4cefb9655f4e4a920f42ad550c13c4. Latest consensus state: current_term: 3 leader_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: false } } }
I20260808 13:04:13.819341 13337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:04:13.821250 13324 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: false } } }
I20260808 13:04:13.821645 13324 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4 [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:13.821946 13337 sys_catalog.cc:455] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "cdbf0bd6947747b9a2f3229974732e17" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39241 } } peers { permanent_uuid: "bd4cefb9655f4e4a920f42ad550c13c4" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 38679 } attrs { promote: false } } peers { permanent_uuid: "31b5517c6b78485b96efad3371700c56" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 36769 } attrs { promote: false } } }
I20260808 13:04:13.822201 13337 sys_catalog.cc:458] T 00000000000000000000000000000000 P 31b5517c6b78485b96efad3371700c56 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:04:13.865772 13350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:13.831560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32980 (local address 127.1.124.131:42219)
0808 13:04:13.848039 (+ 16479us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:13.848049 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:13.853767 (+ 5718us) server_negotiation.cc:407] Connection header received
0808 13:04:13.854597 (+ 830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:13.854602 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:13.854699 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:13.854845 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:13.855927 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:13.858645 (+ 2718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:13.862585 (+ 3940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:13.864363 (+ 1778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:13.864498 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:13.864692 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:13.865263 (+ 571us) server_negotiation.cc:299] Negotiation successful
0808 13:04:13.865404 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16322,"thread_start_us":16509,"threads_started":1}
I20260808 13:04:13.865772 13351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:13.831405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:42219 (local address 127.0.0.1:32980)
0808 13:04:13.853624 (+ 22219us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:13.853675 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:13.853885 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:13.855186 (+ 1301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:13.855192 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:13.855225 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:13.855658 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:13.855675 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:13.858849 (+ 3174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:13.858856 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:13.861291 (+ 2435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:13.861309 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:13.864890 (+ 3581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:13.864899 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:13.865068 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:13.865092 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 13:04:13.865220 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 13:04:13.865365 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22083,"threads_started":1}
I20260808 13:04:13.867312 13094 tablet_service.cc:1467] Processing CreateTablet for tablet 4f052f67a2634e628c067783f6aeedac (DEFAULT_TABLE table=test-table [id=6ccdee5953c4480eb6bc7ad5522389e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:04:13.867846 13094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f052f67a2634e628c067783f6aeedac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:13.871603 13348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:13.831183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55954 (local address 127.1.124.130:45593)
0808 13:04:13.861965 (+ 30782us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:13.861972 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:13.861992 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:04:13.862064 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:13.862069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:13.862158 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:13.862297 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:13.864740 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:13.866700 (+ 1960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:13.869161 (+ 2461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:13.870484 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:13.870900 (+ 416us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:13.871009 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:13.871113 (+ 104us) server_negotiation.cc:299] Negotiation successful
0808 13:04:13.871261 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30623,"thread_start_us":30494,"threads_started":1}
I20260808 13:04:13.871608 13325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:13.830964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:45593 (local address 127.0.0.1:55954)
0808 13:04:13.835871 (+ 4907us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:13.835900 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:13.836080 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:13.862610 (+ 26530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:13.862618 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:13.862660 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:13.863108 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:13.863123 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:13.866833 (+ 3710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:13.866841 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:13.868963 (+ 2122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:13.868980 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:13.870722 (+ 1742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:13.870729 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:13.870949 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:13.870982 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 13:04:13.871090 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 13:04:13.871239 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4740,"mutex_wait_us":16}
I20260808 13:04:13.872941 12853 tablet_service.cc:1467] Processing CreateTablet for tablet 4f052f67a2634e628c067783f6aeedac (DEFAULT_TABLE table=test-table [id=6ccdee5953c4480eb6bc7ad5522389e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:04:13.873483 12853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f052f67a2634e628c067783f6aeedac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:13.875432 13351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:13.832162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:35393 (local address 127.0.0.1:40050)
0808 13:04:13.866051 (+ 33889us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:13.866070 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:13.866188 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:13.866694 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:13.866701 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:13.866726 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:13.866884 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:13.866899 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:13.869506 (+ 2607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:13.869513 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:13.872932 (+ 3419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:13.872945 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:13.874645 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:13.874651 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:13.874804 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:13.874828 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 13:04:13.874930 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 13:04:13.875066 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33739,"mutex_wait_us":24,"thread_start_us":21130,"threads_started":1}
I20260808 13:04:13.880561 13352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:13.832658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40050 (local address 127.1.124.129:35393)
0808 13:04:13.853033 (+ 20375us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:13.853043 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:13.866180 (+ 13137us) server_negotiation.cc:407] Connection header received
0808 13:04:13.866313 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:13.866319 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:13.866429 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:13.866537 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:13.868227 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:13.869292 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:13.873133 (+ 3841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:13.874280 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:13.874456 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:13.874541 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:13.874976 (+ 435us) server_negotiation.cc:299] Negotiation successful
0808 13:04:13.875198 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20173,"thread_start_us":19984,"threads_started":1}
I20260808 13:04:13.881619 12610 tablet_service.cc:1467] Processing CreateTablet for tablet 4f052f67a2634e628c067783f6aeedac (DEFAULT_TABLE table=test-table [id=6ccdee5953c4480eb6bc7ad5522389e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:04:13.882081 12610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4f052f67a2634e628c067783f6aeedac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:13.890628 13360 tablet_bootstrap.cc:492] T 4f052f67a2634e628c067783f6aeedac P 7a9c51158a034c87bc80b34266e50afe: Bootstrap starting.
I20260808 13:04:13.890849 13358 tablet_bootstrap.cc:492] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb: Bootstrap starting.
I20260808 13:04:13.891602 13354 tablet_bootstrap.cc:492] T 4f052f67a2634e628c067783f6aeedac P 068dd3cdc5dd4e8ca38c428773c5d616: Bootstrap starting.
I20260808 13:04:13.892865 13360 tablet_bootstrap.cc:654] T 4f052f67a2634e628c067783f6aeedac P 7a9c51158a034c87bc80b34266e50afe: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:13.893652 13354 tablet_bootstrap.cc:654] T 4f052f67a2634e628c067783f6aeedac P 068dd3cdc5dd4e8ca38c428773c5d616: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:13.895258 13358 tablet_bootstrap.cc:654] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:13.896476 13360 tablet_bootstrap.cc:492] T 4f052f67a2634e628c067783f6aeedac P 7a9c51158a034c87bc80b34266e50afe: No bootstrap required, opened a new log
I20260808 13:04:13.896696 13360 ts_tablet_manager.cc:1397] T 4f052f67a2634e628c067783f6aeedac P 7a9c51158a034c87bc80b34266e50afe: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 13:04:13.897818 13360 raft_consensus.cc:348] T 4f052f67a2634e628c067783f6aeedac P 7a9c51158a034c87bc80b34266e50afe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 45593 } } peers { permanent_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42219 } } peers { permanent_uuid: "7a9c51158a034c87bc80b34266e50afe" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35393 } }
I20260808 13:04:13.898170 13360 raft_consensus.cc:374] T 4f052f67a2634e628c067783f6aeedac P 7a9c51158a034c87bc80b34266e50afe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:13.898300 13360 raft_consensus.cc:729] T 4f052f67a2634e628c067783f6aeedac P 7a9c51158a034c87bc80b34266e50afe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a9c51158a034c87bc80b34266e50afe, State: Initialized, Role: FOLLOWER
I20260808 13:04:13.898645 13360 consensus_queue.cc:260] T 4f052f67a2634e628c067783f6aeedac P 7a9c51158a034c87bc80b34266e50afe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 45593 } } peers { permanent_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42219 } } peers { permanent_uuid: "7a9c51158a034c87bc80b34266e50afe" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35393 } }
I20260808 13:04:13.900946 13360 ts_tablet_manager.cc:1428] T 4f052f67a2634e628c067783f6aeedac P 7a9c51158a034c87bc80b34266e50afe: Time spent starting tablet: real 0.004s user 0.001s sys 0.003s
I20260808 13:04:13.901731 13354 tablet_bootstrap.cc:492] T 4f052f67a2634e628c067783f6aeedac P 068dd3cdc5dd4e8ca38c428773c5d616: No bootstrap required, opened a new log
I20260808 13:04:13.901962 13354 ts_tablet_manager.cc:1397] T 4f052f67a2634e628c067783f6aeedac P 068dd3cdc5dd4e8ca38c428773c5d616: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.007s
I20260808 13:04:13.901989 13358 tablet_bootstrap.cc:492] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb: No bootstrap required, opened a new log
I20260808 13:04:13.902128 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:13.901615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.129:46551)
0808 13:04:13.901831 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:13.901943 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260808 13:04:13.902168 13358 ts_tablet_manager.cc:1397] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260808 13:04:13.902963 13354 raft_consensus.cc:348] T 4f052f67a2634e628c067783f6aeedac P 068dd3cdc5dd4e8ca38c428773c5d616 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 45593 } } peers { permanent_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42219 } } peers { permanent_uuid: "7a9c51158a034c87bc80b34266e50afe" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35393 } }
I20260808 13:04:13.903249 13354 raft_consensus.cc:374] T 4f052f67a2634e628c067783f6aeedac P 068dd3cdc5dd4e8ca38c428773c5d616 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:13.903352 13354 raft_consensus.cc:729] T 4f052f67a2634e628c067783f6aeedac P 068dd3cdc5dd4e8ca38c428773c5d616 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 068dd3cdc5dd4e8ca38c428773c5d616, State: Initialized, Role: FOLLOWER
I20260808 13:04:13.903674 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:13.903189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.129:50539)
0808 13:04:13.903425 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:13.903510 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260808 13:04:13.903625 13354 consensus_queue.cc:260] T 4f052f67a2634e628c067783f6aeedac P 068dd3cdc5dd4e8ca38c428773c5d616 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 45593 } } peers { permanent_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42219 } } peers { permanent_uuid: "7a9c51158a034c87bc80b34266e50afe" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35393 } }
I20260808 13:04:13.904876 13354 ts_tablet_manager.cc:1428] T 4f052f67a2634e628c067783f6aeedac P 068dd3cdc5dd4e8ca38c428773c5d616: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260808 13:04:13.906013 13358 raft_consensus.cc:348] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 45593 } } peers { permanent_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42219 } } peers { permanent_uuid: "7a9c51158a034c87bc80b34266e50afe" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35393 } }
I20260808 13:04:13.906373 13358 raft_consensus.cc:374] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:13.906544 13358 raft_consensus.cc:729] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 001d8fb8a10e470b8ea6e80f680c6ebb, State: Initialized, Role: FOLLOWER
I20260808 13:04:13.906756 12723 heartbeater.cc:503] Master 127.1.124.189:38679 was elected leader, sending a full tablet report...
I20260808 13:04:13.906960 13322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:13.906291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.131:40193)
0808 13:04:13.906689 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:13.906790 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266}
I20260808 13:04:13.906898 13358 consensus_queue.cc:260] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 45593 } } peers { permanent_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42219 } } peers { permanent_uuid: "7a9c51158a034c87bc80b34266e50afe" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35393 } }
I20260808 13:04:13.907903 13196 heartbeater.cc:503] Master 127.1.124.189:38679 was elected leader, sending a full tablet report...
I20260808 13:04:13.910619 13322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:13.909909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.131:60931)
0808 13:04:13.910267 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:13.910401 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210}
I20260808 13:04:13.911537 13358 ts_tablet_manager.cc:1428] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb: Time spent starting tablet: real 0.009s user 0.002s sys 0.004s
I20260808 13:04:13.912451 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:13.911967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.130:47397)
0808 13:04:13.912195 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:13.912301 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260808 13:04:13.913585 12968 heartbeater.cc:503] Master 127.1.124.189:38679 was elected leader, sending a full tablet report...
I20260808 13:04:13.913697 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:13.913245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.130:54391)
0808 13:04:13.913439 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:13.913531 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260808 13:04:13.924029 12724 tablet.cc:2369] T 4f052f67a2634e628c067783f6aeedac P 7a9c51158a034c87bc80b34266e50afe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:04:14.259889 13351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:14.259312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:34542)
0808 13:04:14.259513 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:14.259703 (+ 190us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 13:04:14.260998 13351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:14.260634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:34556)
0808 13:04:14.260812 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:14.260883 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 13:04:14.317904 13369 raft_consensus.cc:482] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:04:14.318279 13369 raft_consensus.cc:504] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 45593 } } peers { permanent_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42219 } } peers { permanent_uuid: "7a9c51158a034c87bc80b34266e50afe" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35393 } }
I20260808 13:04:14.320091 13369 leader_election.cc:288] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 068dd3cdc5dd4e8ca38c428773c5d616 (127.1.124.131:42219), 7a9c51158a034c87bc80b34266e50afe (127.1.124.129:35393)
I20260808 13:04:14.326642 13350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:14.320571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:40989 (local address 127.1.124.131:42219)
0808 13:04:14.320791 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:14.320798 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:14.320886 (+ 88us) server_negotiation.cc:407] Connection header received
0808 13:04:14.321144 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:14.321148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:14.321208 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:14.321323 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:14.321857 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:14.322920 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:14.324773 (+ 1853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:14.325572 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:14.325771 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:14.325850 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:14.326203 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 13:04:14.326355 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 13:04:14.326861 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:14.320475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:42219 (local address 127.1.124.130:40989)
0808 13:04:14.320787 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:14.320814 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:14.320961 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:14.321351 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:14.321356 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:14.321391 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:14.321695 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:14.321705 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:14.323061 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:14.323069 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:14.324606 (+ 1537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:14.324619 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:14.325782 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:14.325788 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:14.325995 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:14.326020 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 13:04:14.326154 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 13:04:14.326324 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260808 13:04:14.326913 13370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:14.320581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:35393 (local address 127.1.124.130:47633)
0808 13:04:14.321385 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:14.321399 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:14.321586 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:14.321957 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:14.321961 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:14.321983 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:14.322261 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:14.322269 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:14.323289 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:14.323295 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:14.324757 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:14.324771 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:14.325947 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:14.325953 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:14.326177 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:14.326210 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 13:04:14.326302 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 13:04:14.326415 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"spinlock_wait_cycles":4864,"thread_start_us":471,"threads_started":1}
I20260808 13:04:14.327349 13352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:14.320569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:47633 (local address 127.1.124.129:35393)
0808 13:04:14.320788 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:14.320796 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:14.321510 (+ 714us) server_negotiation.cc:407] Connection header received
0808 13:04:14.321738 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:14.321748 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:14.321808 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:14.321943 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:14.322412 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:14.323140 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:14.324918 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:14.325694 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:14.325886 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:14.325982 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:14.326887 (+ 905us) server_negotiation.cc:299] Negotiation successful
0808 13:04:14.327079 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 13:04:14.327962 12643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f052f67a2634e628c067783f6aeedac" candidate_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a9c51158a034c87bc80b34266e50afe" is_pre_election: true
I20260808 13:04:14.328334 12643 raft_consensus.cc:2454] T 4f052f67a2634e628c067783f6aeedac P 7a9c51158a034c87bc80b34266e50afe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001d8fb8a10e470b8ea6e80f680c6ebb in term 0.
I20260808 13:04:14.328184 13125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f052f67a2634e628c067783f6aeedac" candidate_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" is_pre_election: true
I20260808 13:04:14.328830 13125 raft_consensus.cc:2454] T 4f052f67a2634e628c067783f6aeedac P 068dd3cdc5dd4e8ca38c428773c5d616 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 001d8fb8a10e470b8ea6e80f680c6ebb in term 0.
I20260808 13:04:14.328958 12777 leader_election.cc:302] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 001d8fb8a10e470b8ea6e80f680c6ebb, 7a9c51158a034c87bc80b34266e50afe; no voters:
I20260808 13:04:14.329417 13369 raft_consensus.cc:2781] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:04:14.329566 13369 raft_consensus.cc:482] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:04:14.329670 13369 raft_consensus.cc:3037] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:14.332131 13369 raft_consensus.cc:504] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 45593 } } peers { permanent_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42219 } } peers { permanent_uuid: "7a9c51158a034c87bc80b34266e50afe" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35393 } }
I20260808 13:04:14.332824 13369 leader_election.cc:288] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb [CANDIDATE]: Term 1 election: Requested vote from peers 068dd3cdc5dd4e8ca38c428773c5d616 (127.1.124.131:42219), 7a9c51158a034c87bc80b34266e50afe (127.1.124.129:35393)
I20260808 13:04:14.333432 13125 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f052f67a2634e628c067783f6aeedac" candidate_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "068dd3cdc5dd4e8ca38c428773c5d616"
I20260808 13:04:14.333519 12643 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4f052f67a2634e628c067783f6aeedac" candidate_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a9c51158a034c87bc80b34266e50afe"
I20260808 13:04:14.333642 13125 raft_consensus.cc:3037] T 4f052f67a2634e628c067783f6aeedac P 068dd3cdc5dd4e8ca38c428773c5d616 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:14.333777 12643 raft_consensus.cc:3037] T 4f052f67a2634e628c067783f6aeedac P 7a9c51158a034c87bc80b34266e50afe [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:14.336007 13125 raft_consensus.cc:2454] T 4f052f67a2634e628c067783f6aeedac P 068dd3cdc5dd4e8ca38c428773c5d616 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001d8fb8a10e470b8ea6e80f680c6ebb in term 1.
I20260808 13:04:14.336201 12643 raft_consensus.cc:2454] T 4f052f67a2634e628c067783f6aeedac P 7a9c51158a034c87bc80b34266e50afe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 001d8fb8a10e470b8ea6e80f680c6ebb in term 1.
I20260808 13:04:14.336627 12778 leader_election.cc:302] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 001d8fb8a10e470b8ea6e80f680c6ebb, 068dd3cdc5dd4e8ca38c428773c5d616, 7a9c51158a034c87bc80b34266e50afe; no voters:
I20260808 13:04:14.336985 13369 raft_consensus.cc:2781] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:04:14.337445 13369 raft_consensus.cc:686] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb [term 1 LEADER]: Becoming Leader. State: Replica: 001d8fb8a10e470b8ea6e80f680c6ebb, State: Running, Role: LEADER
I20260808 13:04:14.337836 13369 consensus_queue.cc:237] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 45593 } } peers { permanent_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42219 } } peers { permanent_uuid: "7a9c51158a034c87bc80b34266e50afe" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35393 } }
I20260808 13:04:14.338757 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:14.338271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.130:38105)
0808 13:04:14.338477 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:14.338581 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 13:04:14.341517 12090 catalog_manager.cc:5495] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb reported cstate change: term changed from 0 to 1, leader changed from <none> to 001d8fb8a10e470b8ea6e80f680c6ebb (127.1.124.130). New cstate: current_term: 1 leader_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "001d8fb8a10e470b8ea6e80f680c6ebb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 45593 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a9c51158a034c87bc80b34266e50afe" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35393 } health_report { overall_health: UNKNOWN } } }
I20260808 13:04:14.422432 13387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:14.416118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55964 (local address 127.1.124.130:45593)
0808 13:04:14.416798 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:14.416802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:14.416819 (+ 17us) server_negotiation.cc:407] Connection header received
0808 13:04:14.416906 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:14.416910 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:14.416986 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:14.417200 (+ 214us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:04:14.417638 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:14.418320 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:14.420253 (+ 1933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:14.420645 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:14.420765 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:14.421232 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:04:14.421805 (+ 573us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:04:14.421921 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:14.422044 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 13:04:14.422113 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":580,"thread_start_us":473,"threads_started":1}
I20260808 13:04:14.426781 12643 raft_consensus.cc:1261] T 4f052f67a2634e628c067783f6aeedac P 7a9c51158a034c87bc80b34266e50afe [term 1 FOLLOWER]: Refusing update from remote peer 001d8fb8a10e470b8ea6e80f680c6ebb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:04:14.427112 13125 raft_consensus.cc:1261] T 4f052f67a2634e628c067783f6aeedac P 068dd3cdc5dd4e8ca38c428773c5d616 [term 1 FOLLOWER]: Refusing update from remote peer 001d8fb8a10e470b8ea6e80f680c6ebb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:04:14.427580 13373 consensus_queue.cc:1035] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a9c51158a034c87bc80b34266e50afe" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:04:14.428004 13369 consensus_queue.cc:1035] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb [LEADER]: Connected to new peer: Peer: permanent_uuid: "068dd3cdc5dd4e8ca38c428773c5d616" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:04:14.433617 13393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:14.430617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.129:41575)
0808 13:04:14.433163 (+ 2546us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:14.433308 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2421,"thread_start_us":2348,"threads_started":1}
I20260808 13:04:14.441138 13388 mvcc.cc:205] Tried to move back new op lower bound from 7316251666124476416 to 7316251665772314624. Current Snapshot: MvccSnapshot[applied={T|T < 7316251666124476416}]
I20260808 13:04:14.443217 13393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:14.442499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.129:53179)
0808 13:04:14.442879 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:14.442997 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236}
I20260808 13:04:14.444736 13396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:14.431988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.131:58671)
0808 13:04:14.444256 (+ 12268us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:14.444403 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12107,"threads_started":1}
I20260808 13:04:14.446769 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:14.446262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.130:42971)
0808 13:04:14.446483 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:14.446580 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260808 13:04:14.456007 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:14.455382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.130:42411)
0808 13:04:14.455657 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:14.455802 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
I20260808 13:04:14.459020 13396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:14.458509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.1.124.131:60557)
0808 13:04:14.458735 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:14.458831 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260808 13:04:14.459694 12090 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:04:14.461007 12090 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37416:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:04:14.467454 12090 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd4cefb9655f4e4a920f42ad550c13c4: Sending DeleteTablet for 3 replicas of tablet 4f052f67a2634e628c067783f6aeedac
I20260808 13:04:14.468739 13094 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f052f67a2634e628c067783f6aeedac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:04:14 UTC) from {username='oryx', principal='oryx/127.1.124.189@KRBTEST.COM'} at 127.0.0.1:32980
I20260808 13:04:14.468743 12853 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f052f67a2634e628c067783f6aeedac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:04:14 UTC) from {username='oryx', principal='oryx/127.1.124.189@KRBTEST.COM'} at 127.0.0.1:55954
I20260808 13:04:14.472806 12610 tablet_service.cc:1514] Processing DeleteTablet for tablet 4f052f67a2634e628c067783f6aeedac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:04:14 UTC) from {username='oryx', principal='oryx/127.1.124.189@KRBTEST.COM'} at 127.0.0.1:40050
I20260808 13:04:14.475368 13408 tablet_replica.cc:330] T 4f052f67a2634e628c067783f6aeedac P 068dd3cdc5dd4e8ca38c428773c5d616: stopping tablet replica
I20260808 13:04:14.475597 13408 raft_consensus.cc:2229] T 4f052f67a2634e628c067783f6aeedac P 068dd3cdc5dd4e8ca38c428773c5d616 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:04:14.475793 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 12369
I20260808 13:04:14.475944 13408 raft_consensus.cc:2258] T 4f052f67a2634e628c067783f6aeedac P 068dd3cdc5dd4e8ca38c428773c5d616 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:04:14.477548 13408 ts_tablet_manager.cc:1905] T 4f052f67a2634e628c067783f6aeedac P 068dd3cdc5dd4e8ca38c428773c5d616: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:14.482316 13408 ts_tablet_manager.cc:1918] T 4f052f67a2634e628c067783f6aeedac P 068dd3cdc5dd4e8ca38c428773c5d616: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:04:14.482466 13408 log.cc:1196] T 4f052f67a2634e628c067783f6aeedac P 068dd3cdc5dd4e8ca38c428773c5d616: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/4f052f67a2634e628c067783f6aeedac
I20260808 13:04:14.482993 13408 ts_tablet_manager.cc:1939] T 4f052f67a2634e628c067783f6aeedac P 068dd3cdc5dd4e8ca38c428773c5d616: Deleting consensus metadata
I20260808 13:04:14.484268 12067 catalog_manager.cc:4841] TS 068dd3cdc5dd4e8ca38c428773c5d616 (127.1.124.131:42219): tablet 4f052f67a2634e628c067783f6aeedac (table test-table [id=6ccdee5953c4480eb6bc7ad5522389e8]) successfully deleted
I20260808 13:04:14.496821 13412 tablet_replica.cc:330] T 4f052f67a2634e628c067783f6aeedac P 7a9c51158a034c87bc80b34266e50afe: stopping tablet replica
I20260808 13:04:14.497097 13412 raft_consensus.cc:2229] T 4f052f67a2634e628c067783f6aeedac P 7a9c51158a034c87bc80b34266e50afe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:04:14.497537 13412 raft_consensus.cc:2258] T 4f052f67a2634e628c067783f6aeedac P 7a9c51158a034c87bc80b34266e50afe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:04:14.498423 13410 tablet_replica.cc:330] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb: stopping tablet replica
I20260808 13:04:14.498695 13410 raft_consensus.cc:2229] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:04:14.499497 13410 raft_consensus.cc:2258] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:04:14.501003 13412 ts_tablet_manager.cc:1905] T 4f052f67a2634e628c067783f6aeedac P 7a9c51158a034c87bc80b34266e50afe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:14.505539 13410 ts_tablet_manager.cc:1905] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:14.514006 13412 ts_tablet_manager.cc:1918] T 4f052f67a2634e628c067783f6aeedac P 7a9c51158a034c87bc80b34266e50afe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:04:14.514166 13412 log.cc:1196] T 4f052f67a2634e628c067783f6aeedac P 7a9c51158a034c87bc80b34266e50afe: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/4f052f67a2634e628c067783f6aeedac
I20260808 13:04:14.514719 13412 ts_tablet_manager.cc:1939] T 4f052f67a2634e628c067783f6aeedac P 7a9c51158a034c87bc80b34266e50afe: Deleting consensus metadata
I20260808 13:04:14.516038 12066 catalog_manager.cc:4841] TS 7a9c51158a034c87bc80b34266e50afe (127.1.124.129:35393): tablet 4f052f67a2634e628c067783f6aeedac (table test-table [id=6ccdee5953c4480eb6bc7ad5522389e8]) successfully deleted
I20260808 13:04:14.522357 13410 ts_tablet_manager.cc:1918] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:04:14.522527 13410 log.cc:1196] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/4f052f67a2634e628c067783f6aeedac
I20260808 13:04:14.523062 13410 ts_tablet_manager.cc:1939] T 4f052f67a2634e628c067783f6aeedac P 001d8fb8a10e470b8ea6e80f680c6ebb: Deleting consensus metadata
I20260808 13:04:14.524638 12067 catalog_manager.cc:4841] TS 001d8fb8a10e470b8ea6e80f680c6ebb (127.1.124.130:45593): tablet 4f052f67a2634e628c067783f6aeedac (table test-table [id=6ccdee5953c4480eb6bc7ad5522389e8]) successfully deleted
I20260808 13:04:14.527000 13352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:14.476761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40064 (local address 127.1.124.129:35393)
0808 13:04:14.476943 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:14.476949 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:14.508714 (+ 31765us) server_negotiation.cc:407] Connection header received
0808 13:04:14.508943 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:14.508949 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:14.509043 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:14.509157 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:14.511202 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:14.512467 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:14.514580 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:14.515206 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:14.515302 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:14.519383 (+ 4081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:14.519404 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:14.519424 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:14.519465 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:14.523569 (+ 4104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:14.524695 (+ 1126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:14.524706 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:14.524709 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:14.524914 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:14.525344 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:14.525349 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:14.525353 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:14.525739 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:14.526058 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:14.526410 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 13:04:14.526641 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 13:04:14.527706 12712 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:14.656688 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 12369
I20260808 13:04:14.683451 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 12745
I20260808 13:04:14.700106 13387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:14.684301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55976 (local address 127.1.124.130:45593)
0808 13:04:14.684529 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:14.684535 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:14.684557 (+ 22us) server_negotiation.cc:407] Connection header received
0808 13:04:14.684818 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:14.684824 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:14.684913 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:14.685041 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:14.686366 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:14.687563 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:14.689002 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:14.689667 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:14.689815 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:14.692491 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:14.692508 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:14.692535 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:14.692578 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:14.696947 (+ 4369us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:14.697726 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:14.697735 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:14.697739 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:14.697893 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:14.698343 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:14.698349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:14.698352 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:14.698815 (+ 463us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:14.699160 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:14.699384 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 13:04:14.699629 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 13:04:14.700958 12953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:14.829222 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 12745
I20260808 13:04:14.858281 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 12979
I20260808 13:04:14.872493 13427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:14.859119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32994 (local address 127.1.124.131:42219)
0808 13:04:14.859597 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:14.859601 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:14.859618 (+ 17us) server_negotiation.cc:407] Connection header received
0808 13:04:14.859674 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:14.859678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:14.859748 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:14.859843 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:14.861446 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:14.862206 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:14.863376 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:14.863771 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:14.863884 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:14.866560 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:14.866577 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:14.866594 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:14.866627 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:14.869690 (+ 3063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:14.870335 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:14.870342 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:14.870345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:14.870489 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:14.870834 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:14.870838 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:14.870840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:14.871208 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:14.871426 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:14.871711 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 13:04:14.871979 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":241,"threads_started":1}
I20260808 13:04:14.873234 13187 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:14.878686 13429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:14.877794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:34572)
0808 13:04:14.878310 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:14.878479 (+ 169us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":260,"threads_started":1}
I20260808 13:04:14.880414 13429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:14.880004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39241 (local address 127.0.0.1:34586)
0808 13:04:14.880202 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:14.880284 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39241: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 13:04:15.009606 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 12979
I20260808 13:04:15.042483 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 11864
I20260808 13:04:15.043684 12132 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:15.172453 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 11864
I20260808 13:04:15.205068 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 12025
I20260808 13:04:15.206295 12460 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:15.329279 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 12025
2026-08-08T13:04:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (12534 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:04:15 dist-test-slave-t3q3 krb5kdc[13438](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:04:15 dist-test-slave-t3q3 krb5kdc[13438](info): set up 2 sockets
Aug 08 13:04:15 dist-test-slave-t3q3 krb5kdc[13438](info): commencing operation
krb5kdc: starting...
W20260808 13:04:16.677115 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.001s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:04:16 dist-test-slave-t3q3 krb5kdc[13438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:04:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:04:16Z Disabled control of system clock
WARNING: no policy specified for oryx/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.190.keytab.
Entry for principal oryx/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.190.keytab.
I20260808 13:04:16.956434 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:36029
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.190.keytab
--principal=oryx/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:38011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:36029
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:17.314054 13457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:17.314409 13457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:17.314505 13457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:17.323506 13457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:04:17.323647 13457 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 13:04:17.323691 13457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:17.323748 13457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:17.323799 13457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:04:17.323863 13457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:04:17.334538 13457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:38011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:36029
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.190.keytab
--principal=oryx/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:36029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:17.335916 13457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:17.337391 13457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:17.350057 13471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:17.349988 13467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:17.350052 13468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:17.350986 13457 server_base.cc:1034] running on GCE node
I20260808 13:04:17.351816 13457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:17.353361 13457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:17.354635 13457 hybrid_clock.cc:648] HybridClock initialized: now 1786194257354559 us; error 43 us; skew 500 ppm
I20260808 13:04:17.355017 13457 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:17 dist-test-slave-t3q3 krb5kdc[13438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194257, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:17.358335 13457 init.cc:377] Logged in from keytab as oryx/127.1.124.190@KRBTEST.COM (short username oryx)
I20260808 13:04:17.359886 13457 webserver.cc:460] Webserver started at http://127.1.124.190:46535/ using document root <none> and password file <none>
I20260808 13:04:17.360688 13457 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:17.360838 13457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:17.361198 13457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:17.364419 13457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "5bdcc074813a402d9f23a14899c0eb44"
format_stamp: "Formatted at 2026-08-08 13:04:17 on dist-test-slave-t3q3"
server_key: "9b86dbf670c9ce1a3a3d8cd500a1c216"
server_key_iv: "b2359e5c1493a100efbe07a31ed73e00"
server_key_version: "encryptionkey@0"
I20260808 13:04:17.365254 13457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "5bdcc074813a402d9f23a14899c0eb44"
format_stamp: "Formatted at 2026-08-08 13:04:17 on dist-test-slave-t3q3"
server_key: "9b86dbf670c9ce1a3a3d8cd500a1c216"
server_key_iv: "b2359e5c1493a100efbe07a31ed73e00"
server_key_version: "encryptionkey@0"
I20260808 13:04:17.370693 13457 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 13:04:17.374329 13484 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:17.375659 13457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 13:04:17.375836 13457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "5bdcc074813a402d9f23a14899c0eb44"
format_stamp: "Formatted at 2026-08-08 13:04:17 on dist-test-slave-t3q3"
server_key: "9b86dbf670c9ce1a3a3d8cd500a1c216"
server_key_iv: "b2359e5c1493a100efbe07a31ed73e00"
server_key_version: "encryptionkey@0"
I20260808 13:04:17.376096 13457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:17.392482 13457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:17.396590 13457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:17.396880 13457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:17.419473 13457 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:36029
I20260808 13:04:17.419485 13583 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:36029 every 8 connection(s)
I20260808 13:04:17.420956 13457 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:04:17.424664 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 13457
I20260808 13:04:17.424925 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:04:17.425004 13584 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:17.425786 1522 external_mini_cluster.cc:1428] Setting key b1acf1dc5ae3e4301017a6ff2a8be83c
I20260808 13:04:17.439967 13584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44: Bootstrap starting.
Aug 08 13:04:17 dist-test-slave-t3q3 krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM
I20260808 13:04:17.443686 13584 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:17.445160 13584 log.cc:824] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44: Log is configured to *not* fsync() on all Append() calls
I20260808 13:04:17.448756 13584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44: No bootstrap required, opened a new log
I20260808 13:04:17.450870 13588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:17.427866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40230 (local address 127.1.124.190:36029)
0808 13:04:17.428763 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:17.428774 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:17.428820 (+ 46us) server_negotiation.cc:407] Connection header received
0808 13:04:17.430362 (+ 1542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:17.430387 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:17.430777 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:17.431106 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:17.433043 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:17.434326 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:17.436159 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:17.436885 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:17.437015 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:17.442095 (+ 5080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:17.442103 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:17.442118 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:17.442153 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:17.445980 (+ 3827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:17.446922 (+ 942us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:17.446930 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:17.446938 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:17.447177 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:17.447564 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:17.447568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:17.447571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:17.448322 (+ 751us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:17.448648 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:17.449081 (+ 433us) server_negotiation.cc:299] Negotiation successful
0808 13:04:17.449526 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":677,"thread_start_us":478,"threads_started":1}
I20260808 13:04:17.457710 13584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bdcc074813a402d9f23a14899c0eb44" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 36029 } }
I20260808 13:04:17.458170 13584 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:17.458288 13584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5bdcc074813a402d9f23a14899c0eb44, State: Initialized, Role: FOLLOWER
I20260808 13:04:17.458899 13584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bdcc074813a402d9f23a14899c0eb44" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 36029 } }
I20260808 13:04:17.459146 13584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:04:17.459290 13584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:04:17.459460 13584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:17.461827 13584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bdcc074813a402d9f23a14899c0eb44" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 36029 } }
I20260808 13:04:17.462435 13584 leader_election.cc:302] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5bdcc074813a402d9f23a14899c0eb44; no voters:
I20260808 13:04:17.463083 13584 leader_election.cc:288] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:04:17.463233 13595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:04:17.463961 13595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44 [term 1 LEADER]: Becoming Leader. State: Replica: 5bdcc074813a402d9f23a14899c0eb44, State: Running, Role: LEADER
I20260808 13:04:17.464571 13595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bdcc074813a402d9f23a14899c0eb44" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 36029 } }
I20260808 13:04:17.465317 13584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:04:17.467487 13597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5bdcc074813a402d9f23a14899c0eb44. Latest consensus state: current_term: 1 leader_uuid: "5bdcc074813a402d9f23a14899c0eb44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bdcc074813a402d9f23a14899c0eb44" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 36029 } } }
I20260808 13:04:17.467460 13596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5bdcc074813a402d9f23a14899c0eb44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5bdcc074813a402d9f23a14899c0eb44" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 36029 } } }
I20260808 13:04:17.468175 13596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44 [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:17.468207 13597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44 [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:17.475546 13601 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:04:17.480742 13601 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:04:17.490103 13601 catalog_manager.cc:1347] Generated new cluster ID: 381893df4a704b559949a0dabfe52111
I20260808 13:04:17.490257 13601 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:04:17.543618 13601 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:04:17.545571 13601 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:04:17.553351 13601 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44: Generated new TSK 0
I20260808 13:04:17.554179 13601 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.1.124.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.124.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.124.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.129.keytab.
Entry for principal oryx/127.1.124.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.129.keytab.
I20260808 13:04:17.624606 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:36029
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.129.keytab
--principal=oryx/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:38011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:17.979876 13622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:17.980254 13622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:17.980443 13622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:17.988130 13622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:17.988253 13622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:17.988340 13622 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 13:04:17.988423 13622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:04:17.998288 13622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:38011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.129.keytab
--principal=oryx/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.124.190:36029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:17.999562 13622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:18.000964 13622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:18.014487 13630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:18.014549 13634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:18.014525 13629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:18.015676 13622 server_base.cc:1034] running on GCE node
I20260808 13:04:18.016407 13622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:18.017421 13622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:18.018678 13622 hybrid_clock.cc:648] HybridClock initialized: now 1786194258018602 us; error 51 us; skew 500 ppm
I20260808 13:04:18.019022 13622 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:18 dist-test-slave-t3q3 krb5kdc[13438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194258, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:18.022444 13622 init.cc:377] Logged in from keytab as oryx/127.1.124.129@KRBTEST.COM (short username oryx)
I20260808 13:04:18.024011 13622 webserver.cc:460] Webserver started at http://127.1.124.129:37803/ using document root <none> and password file <none>
I20260808 13:04:18.024783 13622 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:18.024922 13622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:18.025269 13622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:18.028378 13622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "acf697814b5e4390ae1e3c5faf1a4d62"
format_stamp: "Formatted at 2026-08-08 13:04:18 on dist-test-slave-t3q3"
server_key: "687a832802fa18c1423850eb09f91a85"
server_key_iv: "5e545e9e70ce104a03a1b16919320af0"
server_key_version: "encryptionkey@0"
I20260808 13:04:18.029223 13622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "acf697814b5e4390ae1e3c5faf1a4d62"
format_stamp: "Formatted at 2026-08-08 13:04:18 on dist-test-slave-t3q3"
server_key: "687a832802fa18c1423850eb09f91a85"
server_key_iv: "5e545e9e70ce104a03a1b16919320af0"
server_key_version: "encryptionkey@0"
I20260808 13:04:18.034662 13622 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 13:04:18.038304 13644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:18.039604 13622 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:04:18.039803 13622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "acf697814b5e4390ae1e3c5faf1a4d62"
format_stamp: "Formatted at 2026-08-08 13:04:18 on dist-test-slave-t3q3"
server_key: "687a832802fa18c1423850eb09f91a85"
server_key_iv: "5e545e9e70ce104a03a1b16919320af0"
server_key_version: "encryptionkey@0"
I20260808 13:04:18.040035 13622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:18.062356 13622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:18.067157 13622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:18.067409 13622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:18.070217 13622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:18.070325 13622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:18.070477 13622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:18.070559 13622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 13:04:18 dist-test-slave-t3q3 krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194258, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM
I20260808 13:04:18.095924 13656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:18.070871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:32999 (local address 127.1.124.190:36029)
0808 13:04:18.073647 (+ 2776us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:18.073655 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:18.073694 (+ 39us) server_negotiation.cc:407] Connection header received
0808 13:04:18.075558 (+ 1864us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:18.075565 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:18.075667 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:18.075800 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:18.078131 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:18.079473 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:18.082242 (+ 2769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:18.082858 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:18.083007 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:18.088392 (+ 5385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:18.088398 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:18.088401 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:18.088434 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:18.091900 (+ 3466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:18.092977 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:18.092984 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:18.092988 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:18.093167 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:18.093762 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:18.093770 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:18.093776 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:18.094154 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:18.094323 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:18.095298 (+ 975us) server_negotiation.cc:299] Negotiation successful
0808 13:04:18.095530 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2644,"thread_start_us":2413,"threads_started":1}
I20260808 13:04:18.097561 13657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:18.071639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:36029 (local address 127.1.124.129:32999)
0808 13:04:18.072820 (+ 1181us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:18.072891 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:18.075248 (+ 2357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:18.076045 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:18.076053 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:18.076976 (+ 923us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:18.077843 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:18.077867 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:18.079704 (+ 1837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:18.079711 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:18.082024 (+ 2313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:18.082043 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:18.083121 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:18.083140 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:18.083463 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:18.084418 (+ 955us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:18.084471 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:18.088115 (+ 3644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:18.092052 (+ 3937us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:18.092089 (+ 37us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:18.092110 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:18.092817 (+ 707us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:18.093334 (+ 517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:18.093341 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:18.093347 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:18.093567 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:18.094295 (+ 728us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:18.094308 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:18.094814 (+ 506us) client_negotiation.cc:769] Sending connection context
0808 13:04:18.095180 (+ 366us) client_negotiation.cc:240] Negotiation successful
0808 13:04:18.095830 (+ 650us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":782,"thread_start_us":456,"threads_started":1}
I20260808 13:04:18.112017 13622 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:34901
I20260808 13:04:18.112080 13847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:34901 every 8 connection(s)
I20260808 13:04:18.113678 13622 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:04:18.114576 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 13622
I20260808 13:04:18.114845 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:04:18.116099 13849 heartbeater.cc:347] Connected to a master server at 127.1.124.190:36029
I20260808 13:04:18.116366 13849 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:18.116906 1522 external_mini_cluster.cc:1428] Setting key 4250a90228d032eb68127ac123d330af
I20260808 13:04:18.117267 13849 heartbeater.cc:511] Master 127.1.124.190:36029 requested a full tablet report, sending...
I20260808 13:04:18.119752 13513 ts_manager.cc:194] Registered new tserver with Master: acf697814b5e4390ae1e3c5faf1a4d62 (127.1.124.129:34901)
I20260808 13:04:18.122095 13513 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:32999
I20260808 13:04:18.122659 13823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:18.105056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:36029 (local address 127.1.124.129:55793)
0808 13:04:18.105718 (+ 662us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:18.105735 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:18.105906 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:18.106326 (+ 420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:18.106335 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:18.107249 (+ 914us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:18.107885 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:18.107905 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:18.109547 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:18.109555 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:18.110890 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:18.110907 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:18.111859 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:18.111864 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:18.112015 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:18.112738 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:18.112759 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:18.113648 (+ 889us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:18.118606 (+ 4958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:18.118615 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:18.118622 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:18.119341 (+ 719us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:18.119916 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:18.119925 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:18.119931 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:18.120138 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:18.121085 (+ 947us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:18.121095 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:18.121360 (+ 265us) client_negotiation.cc:769] Sending connection context
0808 13:04:18.121540 (+ 180us) client_negotiation.cc:240] Negotiation successful
0808 13:04:18.121997 (+ 457us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":385,"threads_started":1}
I20260808 13:04:18.123912 13656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:18.104899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:55793 (local address 127.1.124.190:36029)
0808 13:04:18.105093 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:18.105101 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:18.105829 (+ 728us) server_negotiation.cc:407] Connection header received
0808 13:04:18.106047 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:18.106052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:18.106139 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:18.106250 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:18.108129 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:18.109359 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:18.111073 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:18.111532 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:18.111609 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:18.113866 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:18.113873 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:18.113877 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:18.113920 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:18.118399 (+ 4479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:18.119526 (+ 1127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:18.119535 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:18.119541 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:18.119755 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:18.120294 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:18.120302 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:18.120307 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:18.120795 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:18.121005 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:18.123108 (+ 2103us) server_negotiation.cc:299] Negotiation successful
0808 13:04:18.123410 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
WARNING: no policy specified for oryx/127.1.124.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.124.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.124.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.130.keytab.
Entry for principal oryx/127.1.124.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.130.keytab.
I20260808 13:04:18.177532 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:36029
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.130.keytab
--principal=oryx/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:38011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:18.521425 13864 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:18.521710 13864 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:18.521885 13864 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:18.529000 13864 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:18.529107 13864 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:18.529263 13864 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 13:04:18.529393 13864 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:04:18.539016 13864 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:38011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.130.keytab
--principal=oryx/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.124.190:36029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:18.540362 13864 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:18.541671 13864 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:18.555390 13869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:18.555372 13870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:18.555519 13873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:18.556341 13864 server_base.cc:1034] running on GCE node
I20260808 13:04:18.556926 13864 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:18.557816 13864 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:18.559041 13864 hybrid_clock.cc:648] HybridClock initialized: now 1786194258558971 us; error 30 us; skew 500 ppm
I20260808 13:04:18.559305 13864 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:18 dist-test-slave-t3q3 krb5kdc[13438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194258, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:18.562683 13864 init.cc:377] Logged in from keytab as oryx/127.1.124.130@KRBTEST.COM (short username oryx)
I20260808 13:04:18.564193 13864 webserver.cc:460] Webserver started at http://127.1.124.130:46585/ using document root <none> and password file <none>
I20260808 13:04:18.564895 13864 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:18.565024 13864 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:18.565371 13864 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:18.568527 13864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "5daa346ccf3c406c9acda65c08d4fab6"
format_stamp: "Formatted at 2026-08-08 13:04:18 on dist-test-slave-t3q3"
server_key: "b0cdbcff619a80094422fb66ac7ce667"
server_key_iv: "b27b787e6a48393f451f632355ea5e30"
server_key_version: "encryptionkey@0"
I20260808 13:04:18.569350 13864 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "5daa346ccf3c406c9acda65c08d4fab6"
format_stamp: "Formatted at 2026-08-08 13:04:18 on dist-test-slave-t3q3"
server_key: "b0cdbcff619a80094422fb66ac7ce667"
server_key_iv: "b27b787e6a48393f451f632355ea5e30"
server_key_version: "encryptionkey@0"
I20260808 13:04:18.574947 13864 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:04:18.578320 13890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:18.579567 13864 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 13:04:18.579751 13864 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "5daa346ccf3c406c9acda65c08d4fab6"
format_stamp: "Formatted at 2026-08-08 13:04:18 on dist-test-slave-t3q3"
server_key: "b0cdbcff619a80094422fb66ac7ce667"
server_key_iv: "b27b787e6a48393f451f632355ea5e30"
server_key_version: "encryptionkey@0"
I20260808 13:04:18.579977 13864 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:18.596793 13864 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:18.601454 13864 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:18.601783 13864 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:18.604696 13864 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:18.604820 13864 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:18.605014 13864 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:18.605100 13864 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 13:04:18 dist-test-slave-t3q3 krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194258, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM
I20260808 13:04:18.630609 13656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:18.605212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:54093 (local address 127.1.124.190:36029)
0808 13:04:18.605491 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:18.605501 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:18.607029 (+ 1528us) server_negotiation.cc:407] Connection header received
0808 13:04:18.610380 (+ 3351us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:18.610385 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:18.610482 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:18.610609 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:18.612794 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:18.613850 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:18.616502 (+ 2652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:18.617013 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:18.617112 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:18.623605 (+ 6493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:18.623613 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:18.623619 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:18.623667 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:18.627206 (+ 3539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:18.628248 (+ 1042us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:18.628252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:18.628255 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:18.628420 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:18.628872 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:18.628877 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:18.628881 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:18.629173 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:18.629332 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:18.630026 (+ 694us) server_negotiation.cc:299] Negotiation successful
0808 13:04:18.630260 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 13:04:18.631649 13905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:18.605820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:36029 (local address 127.1.124.130:54093)
0808 13:04:18.606810 (+ 990us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:18.606865 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:18.610011 (+ 3146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:18.610900 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:18.610914 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:18.611601 (+ 687us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:18.612569 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:18.612603 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:18.614044 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:18.614048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:18.616276 (+ 2228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:18.616288 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:18.617272 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:18.617284 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:18.617680 (+ 396us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:18.618613 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:18.618664 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:18.623290 (+ 4626us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:18.627341 (+ 4051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:18.627353 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:18.627370 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:18.628055 (+ 685us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:18.628538 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:18.628544 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:18.628548 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:18.628748 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:18.629306 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:18.629314 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:18.629648 (+ 334us) client_negotiation.cc:769] Sending connection context
0808 13:04:18.629914 (+ 266us) client_negotiation.cc:240] Negotiation successful
0808 13:04:18.630429 (+ 515us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":677,"thread_start_us":403,"threads_started":1}
I20260808 13:04:18.653581 13656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:18.638316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:54481 (local address 127.1.124.190:36029)
0808 13:04:18.638561 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:18.638571 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:18.639125 (+ 554us) server_negotiation.cc:407] Connection header received
0808 13:04:18.639272 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:18.639281 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:18.639389 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:18.639531 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:18.640515 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:18.641544 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:18.643530 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:18.644198 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:18.644337 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:18.646679 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:18.646686 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:18.646691 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:18.646729 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:18.649945 (+ 3216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:18.650908 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:18.650916 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:18.650922 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:18.651052 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:18.651477 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:18.651486 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:18.651490 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:18.652188 (+ 698us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:18.652372 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:18.652982 (+ 610us) server_negotiation.cc:299] Negotiation successful
0808 13:04:18.653215 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 13:04:18.653703 14036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:18.638401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:36029 (local address 127.1.124.130:54481)
0808 13:04:18.639020 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:18.639036 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:18.639142 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:18.639539 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:18.639542 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:18.639959 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:18.640365 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:18.640375 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:18.641682 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:18.641689 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:18.643316 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:18.643333 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:18.644568 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:18.644573 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:18.644687 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:18.645399 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:18.645414 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:18.646488 (+ 1074us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:18.650083 (+ 3595us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:18.650088 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:18.650095 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:18.650776 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:18.651167 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:18.651172 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:18.651177 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:18.651308 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:18.652365 (+ 1057us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:18.652376 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:18.652631 (+ 255us) client_negotiation.cc:769] Sending connection context
0808 13:04:18.652872 (+ 241us) client_negotiation.cc:240] Negotiation successful
0808 13:04:18.653238 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":323,"threads_started":1}
I20260808 13:04:18.659396 13864 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:40305
I20260808 13:04:18.659415 14102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:40305 every 8 connection(s)
I20260808 13:04:18.660779 13864 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:04:18.662750 14103 heartbeater.cc:347] Connected to a master server at 127.1.124.190:36029
I20260808 13:04:18.662981 14103 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:18.663601 14103 heartbeater.cc:511] Master 127.1.124.190:36029 requested a full tablet report, sending...
I20260808 13:04:18.664436 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 13864
I20260808 13:04:18.664664 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:04:18.665122 13513 ts_manager.cc:194] Registered new tserver with Master: 5daa346ccf3c406c9acda65c08d4fab6 (127.1.124.130:40305)
I20260808 13:04:18.665256 1522 external_mini_cluster.cc:1428] Setting key 9ae796d54bb0aa236e08d14c8656cc4d
I20260808 13:04:18.665982 13513 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:54093
WARNING: no policy specified for oryx/127.1.124.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.124.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.124.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.131.keytab.
Entry for principal oryx/127.1.124.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.131.keytab.
I20260808 13:04:18.725548 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:36029
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.131.keytab
--principal=oryx/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:38011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:19.074898 14108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:19.075198 14108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:19.075366 14108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:19.082828 14108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:19.082947 14108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:19.083106 14108 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 13:04:19.083231 14108 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:04:19.093708 14108 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:38011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.131.keytab
--principal=oryx/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.1.124.190:36029
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:19.095171 14108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:19.096589 14108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:19.110533 14120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:19.110607 14122 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:19.110545 14124 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:19.112874 14108 server_base.cc:1034] running on GCE node
I20260808 13:04:19.113569 14108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:19.114619 14108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:19.115857 14108 hybrid_clock.cc:648] HybridClock initialized: now 1786194259115783 us; error 53 us; skew 500 ppm
I20260808 13:04:19.116251 14108 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:19 dist-test-slave-t3q3 krb5kdc[13438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194259, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:19.119445 14108 init.cc:377] Logged in from keytab as oryx/127.1.124.131@KRBTEST.COM (short username oryx)
I20260808 13:04:19.120981 14108 webserver.cc:460] Webserver started at http://127.1.124.131:37079/ using document root <none> and password file <none>
I20260808 13:04:19.121685 14108 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:19.121822 14108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:19.122161 14108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:19.125152 14108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "ebee817e0f46434d89c19aea5e53fa04"
format_stamp: "Formatted at 2026-08-08 13:04:19 on dist-test-slave-t3q3"
server_key: "8b285c27f166f685d0cfd36fe9a2a042"
server_key_iv: "5e48cd83c255e8f069067dd653291d03"
server_key_version: "encryptionkey@0"
I20260808 13:04:19.125962 14108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "ebee817e0f46434d89c19aea5e53fa04"
format_stamp: "Formatted at 2026-08-08 13:04:19 on dist-test-slave-t3q3"
server_key: "8b285c27f166f685d0cfd36fe9a2a042"
server_key_iv: "5e48cd83c255e8f069067dd653291d03"
server_key_version: "encryptionkey@0"
I20260808 13:04:19.126374 13849 heartbeater.cc:503] Master 127.1.124.190:36029 was elected leader, sending a full tablet report...
I20260808 13:04:19.131392 14108 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:04:19.135087 14136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:19.136554 14108 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:04:19.136685 14108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "ebee817e0f46434d89c19aea5e53fa04"
format_stamp: "Formatted at 2026-08-08 13:04:19 on dist-test-slave-t3q3"
server_key: "8b285c27f166f685d0cfd36fe9a2a042"
server_key_iv: "5e48cd83c255e8f069067dd653291d03"
server_key_version: "encryptionkey@0"
I20260808 13:04:19.136864 14108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:19.160928 14108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:19.165561 14108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:19.165903 14108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:19.168958 14108 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:19.169080 14108 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:19.169235 14108 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:19.169314 14108 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 13:04:19 dist-test-slave-t3q3 krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194259, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM
I20260808 13:04:19.197023 14151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:19.171011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:39927 (local address 127.1.124.190:36029)
0808 13:04:19.172285 (+ 1274us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:19.172293 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:19.172318 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:04:19.174157 (+ 1839us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:19.174163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:19.174260 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:19.174360 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:19.177163 (+ 2803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:19.178577 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:19.181459 (+ 2882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:19.182264 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:19.182419 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:19.188142 (+ 5723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:19.188151 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:19.188158 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:19.188230 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:19.192535 (+ 4305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:19.193868 (+ 1333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:19.193873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:19.193876 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:19.194039 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:19.194689 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:19.194692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:19.194695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:19.195002 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:19.195145 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:19.196473 (+ 1328us) server_negotiation.cc:299] Negotiation successful
0808 13:04:19.196703 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1151,"thread_start_us":1050,"threads_started":1}
I20260808 13:04:19.199632 14150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:19.170419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:36029 (local address 127.1.124.131:39927)
0808 13:04:19.171315 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:19.171365 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:19.173735 (+ 2370us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:19.174953 (+ 1218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:19.174962 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:19.175907 (+ 945us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:19.176864 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:19.176886 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:19.178765 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:19.178773 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:19.181267 (+ 2494us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:19.181289 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:19.182592 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:19.182602 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:19.182886 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:19.183850 (+ 964us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:19.183899 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:19.187764 (+ 3865us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:19.192746 (+ 4982us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:19.192759 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:19.192790 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:19.193666 (+ 876us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:19.194185 (+ 519us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:19.194193 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:19.194199 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:19.194551 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:19.195150 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:19.195162 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:19.195787 (+ 625us) client_negotiation.cc:769] Sending connection context
0808 13:04:19.196371 (+ 584us) client_negotiation.cc:240] Negotiation successful
0808 13:04:19.197119 (+ 748us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":383,"threads_started":1}
I20260808 13:04:19.217638 14108 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:39001
I20260808 13:04:19.217664 14336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:39001 every 8 connection(s)
I20260808 13:04:19.219034 14108 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:04:19.221426 14339 heartbeater.cc:347] Connected to a master server at 127.1.124.190:36029
I20260808 13:04:19.221678 14339 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:19.222474 14339 heartbeater.cc:511] Master 127.1.124.190:36029 requested a full tablet report, sending...
I20260808 13:04:19.223162 14151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:19.207272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:51821 (local address 127.1.124.190:36029)
0808 13:04:19.207536 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:19.207545 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:19.208535 (+ 990us) server_negotiation.cc:407] Connection header received
0808 13:04:19.208740 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:19.208744 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:19.208822 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:19.208964 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:19.210927 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:19.211659 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:19.213686 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:19.214153 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:19.214299 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:19.217076 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:19.217086 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:19.217092 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:19.217144 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:19.220198 (+ 3054us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:19.221127 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:19.221132 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:19.221135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:19.221330 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:19.221797 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:19.221802 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:19.221805 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:19.222112 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:19.222252 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:19.222753 (+ 501us) server_negotiation.cc:299] Negotiation successful
0808 13:04:19.222932 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260808 13:04:19.223476 14302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:19.207176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:36029 (local address 127.1.124.131:51821)
0808 13:04:19.208375 (+ 1199us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:19.208402 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:19.208605 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:19.209006 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:19.209010 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:19.209836 (+ 826us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:19.210739 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:19.210760 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:19.211818 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:19.211824 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:19.213522 (+ 1698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:19.213547 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:19.214371 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:19.214381 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:19.214618 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:19.215451 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:19.215474 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:19.216765 (+ 1291us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:19.220404 (+ 3639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:19.220411 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:19.220419 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:19.220981 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:19.221462 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:19.221467 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:19.221470 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:19.221655 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:19.222236 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:19.222241 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:19.222450 (+ 209us) client_negotiation.cc:769] Sending connection context
0808 13:04:19.222640 (+ 190us) client_negotiation.cc:240] Negotiation successful
0808 13:04:19.223002 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":986,"thread_start_us":770,"threads_started":1}
I20260808 13:04:19.224154 13515 ts_manager.cc:194] Registered new tserver with Master: ebee817e0f46434d89c19aea5e53fa04 (127.1.124.131:39001)
I20260808 13:04:19.225086 13515 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:39927
I20260808 13:04:19.227517 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 14108
I20260808 13:04:19.227727 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:04:19.228224 1522 external_mini_cluster.cc:1428] Setting key a102760ddb4cdcaffae5f945c3888a68
I20260808 13:04:19.231140 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 13:04:19 dist-test-slave-t3q3 krb5kdc[13438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194259, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 13:04:19 dist-test-slave-t3q3 krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:19 dist-test-slave-t3q3 krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:19.668568 14103 heartbeater.cc:503] Master 127.1.124.190:36029 was elected leader, sending a full tablet report...
I20260808 13:04:20.227809 14339 heartbeater.cc:503] Master 127.1.124.190:36029 was elected leader, sending a full tablet report...
W20260808 13:04:20.401733 14349 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.112s user 0.004s sys 0.016s
W20260808 13:04:20.403153 14151 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:19.283397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40236 (local address 127.1.124.190:36029)
0808 13:04:19.283717 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:19.283727 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:19.284116 (+ 389us) server_negotiation.cc:407] Connection header received
0808 13:04:19.284327 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:19.284331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:19.284403 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:19.284502 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:19.285945 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:19.286741 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:19.287990 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:19.288495 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:19.288605 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:20.402844 (+1114239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40236: BlockingRecv error: recv got EOF from 127.0.0.1:40236 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
Aug 08 13:04:20 dist-test-slave-t3q3 krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194259, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM
I20260808 13:04:20.422890 14151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:20.408010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48996 (local address 127.1.124.190:36029)
0808 13:04:20.408307 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:20.408314 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:20.408497 (+ 183us) server_negotiation.cc:407] Connection header received
0808 13:04:20.408662 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:20.408666 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:20.408734 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:20.408824 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:20.410071 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.410817 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.412043 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.412536 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.412659 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:20.416986 (+ 4327us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:20.416995 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:20.417002 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:20.417067 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:20.419882 (+ 2815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:20.420825 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:20.420829 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:20.420831 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:20.420967 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:20.421439 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:20.421444 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:20.421446 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:20.421793 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:20.422012 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:20.422385 (+ 373us) server_negotiation.cc:299] Negotiation successful
0808 13:04:20.422592 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260808 13:04:20.427963 13515 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48996:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:04:20.435081 13515 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:04:20.477440 14370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:20.448062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:39001 (local address 127.0.0.1:50200)
0808 13:04:20.463646 (+ 15584us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:20.464279 (+ 633us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:20.464585 (+ 306us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:20.465677 (+ 1092us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:20.465690 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:20.465736 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:20.466312 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:20.466333 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.469511 (+ 3178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.469520 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:20.473018 (+ 3498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:20.473047 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.475909 (+ 2862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.475920 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:20.476281 (+ 361us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:20.476371 (+ 90us) client_negotiation.cc:769] Sending connection context
0808 13:04:20.476534 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 13:04:20.476812 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15271,"thread_start_us":15056,"threads_started":1}
I20260808 13:04:20.479051 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:20.448418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50200 (local address 127.1.124.131:39001)
0808 13:04:20.464236 (+ 15818us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:20.464248 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:20.464844 (+ 596us) server_negotiation.cc:407] Connection header received
0808 13:04:20.464979 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:20.464990 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:20.465353 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:20.466585 (+ 1232us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:20.466858 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.469292 (+ 2434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.473354 (+ 4062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.475553 (+ 2199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.478228 (+ 2675us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:20.478465 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:20.478565 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 13:04:20.478775 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15647,"thread_start_us":15491,"threads_started":1}
I20260808 13:04:20.483192 14221 tablet_service.cc:1467] Processing CreateTablet for tablet 19bacd4d78204793ae187f777454ef12 (DEFAULT_TABLE table=test-table [id=d78af71d23f84c218a906849735e9fe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:04:20.485133 14221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19bacd4d78204793ae187f777454ef12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:20.487336 14376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:20.448757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:34901 (local address 127.0.0.1:57314)
0808 13:04:20.478390 (+ 29633us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:20.478414 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:20.478604 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:20.479657 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:20.479662 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:20.479704 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:20.480212 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:20.480227 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.482105 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.482112 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:20.484445 (+ 2333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:20.484458 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.486408 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.486417 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:20.486603 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:20.486641 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 13:04:20.486736 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 13:04:20.486894 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29460,"mutex_wait_us":79,"thread_start_us":29275,"threads_started":1}
I20260808 13:04:20.487717 14370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:20.448393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:40305 (local address 127.0.0.1:51392)
0808 13:04:20.477824 (+ 29431us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:20.477854 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:20.478020 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:20.479482 (+ 1462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:20.479491 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:20.479530 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:20.479807 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:20.479826 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.483116 (+ 3290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.483121 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:20.484989 (+ 1868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:20.485004 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.486955 (+ 1951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.486961 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:20.487166 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:20.487192 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 13:04:20.487246 (+ 54us) client_negotiation.cc:240] Negotiation successful
0808 13:04:20.487343 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29287,"mutex_wait_us":2,"thread_start_us":34254,"threads_started":1}
I20260808 13:04:20.488377 14378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:20.449600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57314 (local address 127.1.124.129:34901)
0808 13:04:20.477125 (+ 27525us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:20.477140 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:20.478580 (+ 1440us) server_negotiation.cc:407] Connection header received
0808 13:04:20.478793 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:20.478802 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:20.479092 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:20.479379 (+ 287us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:20.480453 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.481922 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.484612 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.486169 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.486324 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:20.486532 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:20.487844 (+ 1312us) server_negotiation.cc:299] Negotiation successful
0808 13:04:20.488037 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27294,"thread_start_us":27557,"threads_started":1}
I20260808 13:04:20.488754 14377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:20.449072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51392 (local address 127.1.124.130:40305)
0808 13:04:20.477857 (+ 28785us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:20.477866 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:20.478785 (+ 919us) server_negotiation.cc:407] Connection header received
0808 13:04:20.478935 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:20.478947 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:20.479221 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:20.480034 (+ 813us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:20.480321 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.482299 (+ 1978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.485267 (+ 2968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.486762 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.487981 (+ 1219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:20.488220 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:20.488274 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 13:04:20.488392 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28620,"thread_start_us":33734,"threads_started":1}
I20260808 13:04:20.492013 13735 tablet_service.cc:1467] Processing CreateTablet for tablet 19bacd4d78204793ae187f777454ef12 (DEFAULT_TABLE table=test-table [id=d78af71d23f84c218a906849735e9fe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:04:20.492246 13978 tablet_service.cc:1467] Processing CreateTablet for tablet 19bacd4d78204793ae187f777454ef12 (DEFAULT_TABLE table=test-table [id=d78af71d23f84c218a906849735e9fe6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:04:20.493391 13735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19bacd4d78204793ae187f777454ef12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:20.493561 13978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19bacd4d78204793ae187f777454ef12. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:20.501945 14379 tablet_bootstrap.cc:492] T 19bacd4d78204793ae187f777454ef12 P ebee817e0f46434d89c19aea5e53fa04: Bootstrap starting.
I20260808 13:04:20.506063 14379 tablet_bootstrap.cc:654] T 19bacd4d78204793ae187f777454ef12 P ebee817e0f46434d89c19aea5e53fa04: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:20.507218 14382 tablet_bootstrap.cc:492] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62: Bootstrap starting.
I20260808 13:04:20.507776 14379 log.cc:824] T 19bacd4d78204793ae187f777454ef12 P ebee817e0f46434d89c19aea5e53fa04: Log is configured to *not* fsync() on all Append() calls
I20260808 13:04:20.512054 14382 tablet_bootstrap.cc:654] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:20.513016 14383 tablet_bootstrap.cc:492] T 19bacd4d78204793ae187f777454ef12 P 5daa346ccf3c406c9acda65c08d4fab6: Bootstrap starting.
I20260808 13:04:20.514389 14382 log.cc:824] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62: Log is configured to *not* fsync() on all Append() calls
I20260808 13:04:20.517189 14383 tablet_bootstrap.cc:654] T 19bacd4d78204793ae187f777454ef12 P 5daa346ccf3c406c9acda65c08d4fab6: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:20.517520 14379 tablet_bootstrap.cc:492] T 19bacd4d78204793ae187f777454ef12 P ebee817e0f46434d89c19aea5e53fa04: No bootstrap required, opened a new log
I20260808 13:04:20.517819 14379 ts_tablet_manager.cc:1397] T 19bacd4d78204793ae187f777454ef12 P ebee817e0f46434d89c19aea5e53fa04: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.005s
I20260808 13:04:20.517817 14382 tablet_bootstrap.cc:492] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62: No bootstrap required, opened a new log
I20260808 13:04:20.518102 14382 ts_tablet_manager.cc:1397] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260808 13:04:20.518517 14383 log.cc:824] T 19bacd4d78204793ae187f777454ef12 P 5daa346ccf3c406c9acda65c08d4fab6: Log is configured to *not* fsync() on all Append() calls
I20260808 13:04:20.521603 14383 tablet_bootstrap.cc:492] T 19bacd4d78204793ae187f777454ef12 P 5daa346ccf3c406c9acda65c08d4fab6: No bootstrap required, opened a new log
I20260808 13:04:20.521862 14383 ts_tablet_manager.cc:1397] T 19bacd4d78204793ae187f777454ef12 P 5daa346ccf3c406c9acda65c08d4fab6: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 13:04:20.525858 14379 raft_consensus.cc:348] T 19bacd4d78204793ae187f777454ef12 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } }
I20260808 13:04:20.526230 14379 raft_consensus.cc:374] T 19bacd4d78204793ae187f777454ef12 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:20.526324 14379 raft_consensus.cc:729] T 19bacd4d78204793ae187f777454ef12 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebee817e0f46434d89c19aea5e53fa04, State: Initialized, Role: FOLLOWER
I20260808 13:04:20.526952 14379 consensus_queue.cc:260] T 19bacd4d78204793ae187f777454ef12 P ebee817e0f46434d89c19aea5e53fa04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } }
I20260808 13:04:20.528609 14379 ts_tablet_manager.cc:1428] T 19bacd4d78204793ae187f777454ef12 P ebee817e0f46434d89c19aea5e53fa04: Time spent starting tablet: real 0.011s user 0.008s sys 0.004s
I20260808 13:04:20.528478 14382 raft_consensus.cc:348] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } }
I20260808 13:04:20.528910 14382 raft_consensus.cc:374] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:20.529022 14382 raft_consensus.cc:729] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf697814b5e4390ae1e3c5faf1a4d62, State: Initialized, Role: FOLLOWER
I20260808 13:04:20.529616 14382 consensus_queue.cc:260] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } }
I20260808 13:04:20.531339 14382 ts_tablet_manager.cc:1428] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260808 13:04:20.533595 14383 raft_consensus.cc:348] T 19bacd4d78204793ae187f777454ef12 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } }
I20260808 13:04:20.534063 14383 raft_consensus.cc:374] T 19bacd4d78204793ae187f777454ef12 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:20.534188 14383 raft_consensus.cc:729] T 19bacd4d78204793ae187f777454ef12 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5daa346ccf3c406c9acda65c08d4fab6, State: Initialized, Role: FOLLOWER
I20260808 13:04:20.534845 14383 consensus_queue.cc:260] T 19bacd4d78204793ae187f777454ef12 P 5daa346ccf3c406c9acda65c08d4fab6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } }
I20260808 13:04:20.536422 14383 ts_tablet_manager.cc:1428] T 19bacd4d78204793ae187f777454ef12 P 5daa346ccf3c406c9acda65c08d4fab6: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260808 13:04:20.600363 14389 raft_consensus.cc:482] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:04:20.600759 14389 raft_consensus.cc:504] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } }
I20260808 13:04:20.602690 14389 leader_election.cc:288] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ebee817e0f46434d89c19aea5e53fa04 (127.1.124.131:39001), 5daa346ccf3c406c9acda65c08d4fab6 (127.1.124.130:40305)
I20260808 13:04:20.612605 14377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:20.603152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:32849 (local address 127.1.124.130:40305)
0808 13:04:20.603413 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:20.603422 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:20.604373 (+ 951us) server_negotiation.cc:407] Connection header received
0808 13:04:20.604574 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:20.604579 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:20.604661 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:20.604766 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:20.605412 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.606426 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.609982 (+ 3556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.611324 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.611490 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:20.611569 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:20.612207 (+ 638us) server_negotiation.cc:299] Negotiation successful
0808 13:04:20.612346 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 13:04:20.612607 14371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:20.603189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:45913 (local address 127.1.124.131:39001)
0808 13:04:20.603660 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:20.603669 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:20.604010 (+ 341us) server_negotiation.cc:407] Connection header received
0808 13:04:20.604265 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:20.604270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:20.604354 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:20.604483 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:20.605132 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.606291 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.609991 (+ 3700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.611311 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.611495 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:20.611569 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:20.612207 (+ 638us) server_negotiation.cc:299] Negotiation successful
0808 13:04:20.612346 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262}
I20260808 13:04:20.612881 14391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:20.603008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:40305 (local address 127.1.124.129:32849)
0808 13:04:20.604237 (+ 1229us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:20.604255 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:20.604399 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:20.604784 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:20.604790 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:20.604813 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:20.605256 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:20.605271 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.606553 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.606561 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:20.609764 (+ 3203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:20.609784 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.611615 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.611623 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:20.611952 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:20.612020 (+ 68us) client_negotiation.cc:769] Sending connection context
0808 13:04:20.612162 (+ 142us) client_negotiation.cc:240] Negotiation successful
0808 13:04:20.612343 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1083,"mutex_wait_us":287,"thread_start_us":444,"threads_started":1}
I20260808 13:04:20.613365 14392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:20.602908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:39001 (local address 127.1.124.129:45913)
0808 13:04:20.603872 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:20.603898 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:20.604107 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:20.604498 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:20.604506 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:20.604538 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:20.604959 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:20.604974 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.606529 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.606534 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:20.609773 (+ 3239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:20.609784 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.611635 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.611640 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:20.611954 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:20.612020 (+ 66us) client_negotiation.cc:769] Sending connection context
0808 13:04:20.612157 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 13:04:20.612343 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"mutex_wait_us":373,"thread_start_us":932,"threads_started":1}
I20260808 13:04:20.614101 14014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19bacd4d78204793ae187f777454ef12" candidate_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5daa346ccf3c406c9acda65c08d4fab6" is_pre_election: true
I20260808 13:04:20.614689 14014 raft_consensus.cc:2454] T 19bacd4d78204793ae187f777454ef12 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf697814b5e4390ae1e3c5faf1a4d62 in term 0.
I20260808 13:04:20.615351 13647 leader_election.cc:302] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5daa346ccf3c406c9acda65c08d4fab6, acf697814b5e4390ae1e3c5faf1a4d62; no voters:
I20260808 13:04:20.615345 14250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19bacd4d78204793ae187f777454ef12" candidate_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebee817e0f46434d89c19aea5e53fa04" is_pre_election: true
I20260808 13:04:20.615829 14389 raft_consensus.cc:2781] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:04:20.615983 14250 raft_consensus.cc:2454] T 19bacd4d78204793ae187f777454ef12 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf697814b5e4390ae1e3c5faf1a4d62 in term 0.
I20260808 13:04:20.616093 14389 raft_consensus.cc:482] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:04:20.616220 14389 raft_consensus.cc:3037] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:20.618718 14389 raft_consensus.cc:504] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } }
I20260808 13:04:20.619403 14389 leader_election.cc:288] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62 [CANDIDATE]: Term 1 election: Requested vote from peers ebee817e0f46434d89c19aea5e53fa04 (127.1.124.131:39001), 5daa346ccf3c406c9acda65c08d4fab6 (127.1.124.130:40305)
I20260808 13:04:20.619843 14250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19bacd4d78204793ae187f777454ef12" candidate_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebee817e0f46434d89c19aea5e53fa04"
I20260808 13:04:20.620133 14250 raft_consensus.cc:3037] T 19bacd4d78204793ae187f777454ef12 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:20.620074 14014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19bacd4d78204793ae187f777454ef12" candidate_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5daa346ccf3c406c9acda65c08d4fab6"
I20260808 13:04:20.620385 14014 raft_consensus.cc:3037] T 19bacd4d78204793ae187f777454ef12 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:20.622565 14014 raft_consensus.cc:2454] T 19bacd4d78204793ae187f777454ef12 P 5daa346ccf3c406c9acda65c08d4fab6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf697814b5e4390ae1e3c5faf1a4d62 in term 1.
I20260808 13:04:20.622570 14250 raft_consensus.cc:2454] T 19bacd4d78204793ae187f777454ef12 P ebee817e0f46434d89c19aea5e53fa04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf697814b5e4390ae1e3c5faf1a4d62 in term 1.
I20260808 13:04:20.623061 13648 leader_election.cc:302] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: acf697814b5e4390ae1e3c5faf1a4d62, ebee817e0f46434d89c19aea5e53fa04; no voters:
I20260808 13:04:20.623440 14389 raft_consensus.cc:2781] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:04:20.623937 14389 raft_consensus.cc:686] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62 [term 1 LEADER]: Becoming Leader. State: Replica: acf697814b5e4390ae1e3c5faf1a4d62, State: Running, Role: LEADER
I20260808 13:04:20.624547 14389 consensus_queue.cc:237] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } }
W20260808 13:04:20.628737 13852 tablet.cc:2369] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:04:20.629024 13513 catalog_manager.cc:5495] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf697814b5e4390ae1e3c5faf1a4d62 (127.1.124.129). New cstate: current_term: 1 leader_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } health_report { overall_health: HEALTHY } } }
I20260808 13:04:20.648651 14378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:20.642835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57320 (local address 127.1.124.129:34901)
0808 13:04:20.643042 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:20.643046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:20.643065 (+ 19us) server_negotiation.cc:407] Connection header received
0808 13:04:20.643251 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:20.643254 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:20.643331 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:20.643460 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:04:20.643910 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.644758 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.646359 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:20.646849 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:20.646967 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:20.647439 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:04:20.648029 (+ 590us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:04:20.648252 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:20.648338 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 13:04:20.648415 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 13:04:20.657502 14250 raft_consensus.cc:1261] T 19bacd4d78204793ae187f777454ef12 P ebee817e0f46434d89c19aea5e53fa04 [term 1 FOLLOWER]: Refusing update from remote peer acf697814b5e4390ae1e3c5faf1a4d62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:04:20.657505 14014 raft_consensus.cc:1261] T 19bacd4d78204793ae187f777454ef12 P 5daa346ccf3c406c9acda65c08d4fab6 [term 1 FOLLOWER]: Refusing update from remote peer acf697814b5e4390ae1e3c5faf1a4d62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:04:20.658617 14393 consensus_queue.cc:1035] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:04:20.659453 14389 consensus_queue.cc:1035] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:04:20.664148 14104 tablet.cc:2369] T 19bacd4d78204793ae187f777454ef12 P 5daa346ccf3c406c9acda65c08d4fab6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:04:20.680459 14402 mvcc.cc:205] Tried to move back new op lower bound from 7316251691642519552 to 7316251691521785856. Current Snapshot: MvccSnapshot[applied={T|T < 7316251691642519552}]
I20260808 13:04:20.694873 13513 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:04:20.695184 13513 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:04:20.699430 13513 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44: Sending DeleteTablet for 3 replicas of tablet 19bacd4d78204793ae187f777454ef12
I20260808 13:04:20.700904 13978 tablet_service.cc:1514] Processing DeleteTablet for tablet 19bacd4d78204793ae187f777454ef12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:04:20 UTC) from {username='oryx', principal='oryx/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:51392
I20260808 13:04:20.700974 14221 tablet_service.cc:1514] Processing DeleteTablet for tablet 19bacd4d78204793ae187f777454ef12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:04:20 UTC) from {username='oryx', principal='oryx/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:50200
I20260808 13:04:20.701504 13735 tablet_service.cc:1514] Processing DeleteTablet for tablet 19bacd4d78204793ae187f777454ef12 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:04:20 UTC) from {username='oryx', principal='oryx/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:57314
I20260808 13:04:20.704106 13513 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48996:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:04:20.706281 13513 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:04:20.714372 14417 tablet_replica.cc:330] T 19bacd4d78204793ae187f777454ef12 P ebee817e0f46434d89c19aea5e53fa04: stopping tablet replica
I20260808 13:04:20.715044 14417 raft_consensus.cc:2229] T 19bacd4d78204793ae187f777454ef12 P ebee817e0f46434d89c19aea5e53fa04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:04:20.715749 14417 raft_consensus.cc:2258] T 19bacd4d78204793ae187f777454ef12 P ebee817e0f46434d89c19aea5e53fa04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:04:20.719173 13734 tablet_service.cc:1467] Processing CreateTablet for tablet e4c85a6e0a3a40798885880c1fea77e0 (DEFAULT_TABLE table=test-table [id=85c69886fd4843818e0f2b621befd594]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:04:20.719637 14221 tablet_service.cc:1467] Processing CreateTablet for tablet e4c85a6e0a3a40798885880c1fea77e0 (DEFAULT_TABLE table=test-table [id=85c69886fd4843818e0f2b621befd594]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:04:20.720002 13734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4c85a6e0a3a40798885880c1fea77e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:20.720324 14221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4c85a6e0a3a40798885880c1fea77e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:20.722688 14418 tablet_replica.cc:330] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62: stopping tablet replica
I20260808 13:04:20.723193 14418 raft_consensus.cc:2229] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:04:20.723881 14418 raft_consensus.cc:2258] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:04:20.727936 14379 tablet_bootstrap.cc:492] T e4c85a6e0a3a40798885880c1fea77e0 P ebee817e0f46434d89c19aea5e53fa04: Bootstrap starting.
I20260808 13:04:20.728260 14382 tablet_bootstrap.cc:492] T e4c85a6e0a3a40798885880c1fea77e0 P acf697814b5e4390ae1e3c5faf1a4d62: Bootstrap starting.
I20260808 13:04:20.729138 13978 tablet_service.cc:1467] Processing CreateTablet for tablet e4c85a6e0a3a40798885880c1fea77e0 (DEFAULT_TABLE table=test-table [id=85c69886fd4843818e0f2b621befd594]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:04:20.729724 14379 tablet_bootstrap.cc:654] T e4c85a6e0a3a40798885880c1fea77e0 P ebee817e0f46434d89c19aea5e53fa04: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:20.729534 14416 tablet_replica.cc:330] T 19bacd4d78204793ae187f777454ef12 P 5daa346ccf3c406c9acda65c08d4fab6: stopping tablet replica
I20260808 13:04:20.730100 14382 tablet_bootstrap.cc:654] T e4c85a6e0a3a40798885880c1fea77e0 P acf697814b5e4390ae1e3c5faf1a4d62: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:20.730473 14416 raft_consensus.cc:2229] T 19bacd4d78204793ae187f777454ef12 P 5daa346ccf3c406c9acda65c08d4fab6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:04:20.731416 14416 raft_consensus.cc:2258] T 19bacd4d78204793ae187f777454ef12 P 5daa346ccf3c406c9acda65c08d4fab6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:04:20.733285 14417 ts_tablet_manager.cc:1905] T 19bacd4d78204793ae187f777454ef12 P ebee817e0f46434d89c19aea5e53fa04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:20.730873 13978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4c85a6e0a3a40798885880c1fea77e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:20.727672 14418 ts_tablet_manager.cc:1905] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:20.738687 14416 ts_tablet_manager.cc:1905] T 19bacd4d78204793ae187f777454ef12 P 5daa346ccf3c406c9acda65c08d4fab6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:20.751055 14417 ts_tablet_manager.cc:1918] T 19bacd4d78204793ae187f777454ef12 P ebee817e0f46434d89c19aea5e53fa04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:04:20.751552 14383 tablet_bootstrap.cc:492] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6: Bootstrap starting.
I20260808 13:04:20.753564 14383 tablet_bootstrap.cc:654] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:20.754230 14382 tablet_bootstrap.cc:492] T e4c85a6e0a3a40798885880c1fea77e0 P acf697814b5e4390ae1e3c5faf1a4d62: No bootstrap required, opened a new log
I20260808 13:04:20.754410 14382 ts_tablet_manager.cc:1397] T e4c85a6e0a3a40798885880c1fea77e0 P acf697814b5e4390ae1e3c5faf1a4d62: Time spent bootstrapping tablet: real 0.026s user 0.006s sys 0.004s
I20260808 13:04:20.755555 14382 raft_consensus.cc:348] T e4c85a6e0a3a40798885880c1fea77e0 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:20.755911 14382 raft_consensus.cc:374] T e4c85a6e0a3a40798885880c1fea77e0 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:20.756044 14382 raft_consensus.cc:729] T e4c85a6e0a3a40798885880c1fea77e0 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf697814b5e4390ae1e3c5faf1a4d62, State: Initialized, Role: FOLLOWER
I20260808 13:04:20.756493 14382 consensus_queue.cc:260] T e4c85a6e0a3a40798885880c1fea77e0 P acf697814b5e4390ae1e3c5faf1a4d62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:20.760283 14417 log.cc:1196] T 19bacd4d78204793ae187f777454ef12 P ebee817e0f46434d89c19aea5e53fa04: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/19bacd4d78204793ae187f777454ef12
I20260808 13:04:20.760550 14382 ts_tablet_manager.cc:1428] T e4c85a6e0a3a40798885880c1fea77e0 P acf697814b5e4390ae1e3c5faf1a4d62: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260808 13:04:20.762410 14417 ts_tablet_manager.cc:1939] T 19bacd4d78204793ae187f777454ef12 P ebee817e0f46434d89c19aea5e53fa04: Deleting consensus metadata
I20260808 13:04:20.764945 14379 tablet_bootstrap.cc:492] T e4c85a6e0a3a40798885880c1fea77e0 P ebee817e0f46434d89c19aea5e53fa04: No bootstrap required, opened a new log
I20260808 13:04:20.765125 14379 ts_tablet_manager.cc:1397] T e4c85a6e0a3a40798885880c1fea77e0 P ebee817e0f46434d89c19aea5e53fa04: Time spent bootstrapping tablet: real 0.037s user 0.016s sys 0.011s
I20260808 13:04:20.766038 14379 raft_consensus.cc:348] T e4c85a6e0a3a40798885880c1fea77e0 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:20.766287 14379 raft_consensus.cc:374] T e4c85a6e0a3a40798885880c1fea77e0 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:20.766383 14379 raft_consensus.cc:729] T e4c85a6e0a3a40798885880c1fea77e0 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebee817e0f46434d89c19aea5e53fa04, State: Initialized, Role: FOLLOWER
I20260808 13:04:20.766649 14379 consensus_queue.cc:260] T e4c85a6e0a3a40798885880c1fea77e0 P ebee817e0f46434d89c19aea5e53fa04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:20.767338 14379 ts_tablet_manager.cc:1428] T e4c85a6e0a3a40798885880c1fea77e0 P ebee817e0f46434d89c19aea5e53fa04: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260808 13:04:20.768348 14418 ts_tablet_manager.cc:1918] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:04:20.768533 14418 log.cc:1196] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/19bacd4d78204793ae187f777454ef12
I20260808 13:04:20.769091 14418 ts_tablet_manager.cc:1939] T 19bacd4d78204793ae187f777454ef12 P acf697814b5e4390ae1e3c5faf1a4d62: Deleting consensus metadata
I20260808 13:04:20.770799 14416 ts_tablet_manager.cc:1918] T 19bacd4d78204793ae187f777454ef12 P 5daa346ccf3c406c9acda65c08d4fab6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:04:20.770989 14416 log.cc:1196] T 19bacd4d78204793ae187f777454ef12 P 5daa346ccf3c406c9acda65c08d4fab6: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/19bacd4d78204793ae187f777454ef12
I20260808 13:04:20.771065 13489 catalog_manager.cc:4841] TS acf697814b5e4390ae1e3c5faf1a4d62 (127.1.124.129:34901): tablet 19bacd4d78204793ae187f777454ef12 (table test-table [id=d78af71d23f84c218a906849735e9fe6]) successfully deleted
I20260808 13:04:20.771584 14416 ts_tablet_manager.cc:1939] T 19bacd4d78204793ae187f777454ef12 P 5daa346ccf3c406c9acda65c08d4fab6: Deleting consensus metadata
I20260808 13:04:20.772416 14383 tablet_bootstrap.cc:492] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6: No bootstrap required, opened a new log
I20260808 13:04:20.772600 14383 ts_tablet_manager.cc:1397] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.005s
I20260808 13:04:20.773692 13492 catalog_manager.cc:4841] TS 5daa346ccf3c406c9acda65c08d4fab6 (127.1.124.130:40305): tablet 19bacd4d78204793ae187f777454ef12 (table test-table [id=d78af71d23f84c218a906849735e9fe6]) successfully deleted
I20260808 13:04:20.773618 14383 raft_consensus.cc:348] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:20.773864 14383 raft_consensus.cc:374] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:20.773970 14383 raft_consensus.cc:729] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5daa346ccf3c406c9acda65c08d4fab6, State: Initialized, Role: FOLLOWER
I20260808 13:04:20.774148 13493 catalog_manager.cc:4841] TS ebee817e0f46434d89c19aea5e53fa04 (127.1.124.131:39001): tablet 19bacd4d78204793ae187f777454ef12 (table test-table [id=d78af71d23f84c218a906849735e9fe6]) successfully deleted
I20260808 13:04:20.774289 14383 consensus_queue.cc:260] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:20.774845 14383 ts_tablet_manager.cc:1428] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
W20260808 13:04:20.972173 14342 tablet.cc:2369] T e4c85a6e0a3a40798885880c1fea77e0 P ebee817e0f46434d89c19aea5e53fa04: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:04:21.195086 14390 raft_consensus.cc:482] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:04:21.195376 14390 raft_consensus.cc:504] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:21.197181 14390 leader_election.cc:288] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf697814b5e4390ae1e3c5faf1a4d62 (127.1.124.129:34901), ebee817e0f46434d89c19aea5e53fa04 (127.1.124.131:39001)
I20260808 13:04:21.217455 14429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:21.197419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:34901 (local address 127.1.124.130:37481)
0808 13:04:21.209683 (+ 12264us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:21.209726 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:21.209974 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:21.210585 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:21.210590 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:21.210619 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:21.210916 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:21.210926 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.212452 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.212456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:21.214767 (+ 2311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:21.214784 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.216440 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.216450 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:21.216686 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:21.216774 (+ 88us) client_negotiation.cc:769] Sending connection context
0808 13:04:21.216875 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 13:04:21.217037 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12045,"mutex_wait_us":25,"thread_start_us":12999,"threads_started":1}
I20260808 13:04:21.217917 14431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:21.197759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:51983 (local address 127.1.124.131:39001)
0808 13:04:21.210562 (+ 12803us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:21.210569 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:21.210593 (+ 24us) server_negotiation.cc:407] Connection header received
0808 13:04:21.210677 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:21.210683 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:21.210760 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:21.210856 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:21.211505 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.212650 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.215221 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.216365 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.216549 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:21.216641 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:21.217150 (+ 509us) server_negotiation.cc:299] Negotiation successful
0808 13:04:21.217325 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12646,"thread_start_us":12533,"threads_started":1}
I20260808 13:04:21.218174 14425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:21.197700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:39001 (local address 127.1.124.130:51983)
0808 13:04:21.210303 (+ 12603us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:21.210322 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:21.210428 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:21.210917 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:21.210921 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:21.210942 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:21.211302 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:21.211310 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.212792 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.212797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:21.215011 (+ 2214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:21.215024 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.216609 (+ 1585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.216615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:21.216802 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:21.216833 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 13:04:21.216935 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 13:04:21.217107 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12469,"mutex_wait_us":99,"thread_start_us":11567,"threads_started":1}
I20260808 13:04:21.218349 14427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:21.197575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:37481 (local address 127.1.124.129:34901)
0808 13:04:21.210019 (+ 12444us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:21.210029 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:21.210067 (+ 38us) server_negotiation.cc:407] Connection header received
0808 13:04:21.210159 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:21.210164 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:21.210253 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:21.210387 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:21.211089 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.212271 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.214983 (+ 2712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.216254 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.216401 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:21.216494 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:21.217902 (+ 1408us) server_negotiation.cc:299] Negotiation successful
0808 13:04:21.218061 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12300,"thread_start_us":12188,"threads_started":1}
I20260808 13:04:21.219056 14250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4c85a6e0a3a40798885880c1fea77e0" candidate_uuid: "5daa346ccf3c406c9acda65c08d4fab6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebee817e0f46434d89c19aea5e53fa04" is_pre_election: true
I20260808 13:04:21.219331 14250 raft_consensus.cc:2454] T e4c85a6e0a3a40798885880c1fea77e0 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5daa346ccf3c406c9acda65c08d4fab6 in term 0.
I20260808 13:04:21.219192 13767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4c85a6e0a3a40798885880c1fea77e0" candidate_uuid: "5daa346ccf3c406c9acda65c08d4fab6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" is_pre_election: true
I20260808 13:04:21.219609 13767 raft_consensus.cc:2454] T e4c85a6e0a3a40798885880c1fea77e0 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5daa346ccf3c406c9acda65c08d4fab6 in term 0.
I20260808 13:04:21.219859 13896 leader_election.cc:302] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5daa346ccf3c406c9acda65c08d4fab6, ebee817e0f46434d89c19aea5e53fa04; no voters:
I20260808 13:04:21.220355 14390 raft_consensus.cc:2781] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:04:21.220491 14390 raft_consensus.cc:482] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:04:21.220594 14390 raft_consensus.cc:3037] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:21.222776 14390 raft_consensus.cc:504] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:21.223467 14390 leader_election.cc:288] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6 [CANDIDATE]: Term 1 election: Requested vote from peers acf697814b5e4390ae1e3c5faf1a4d62 (127.1.124.129:34901), ebee817e0f46434d89c19aea5e53fa04 (127.1.124.131:39001)
I20260808 13:04:21.223953 13767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4c85a6e0a3a40798885880c1fea77e0" candidate_uuid: "5daa346ccf3c406c9acda65c08d4fab6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf697814b5e4390ae1e3c5faf1a4d62"
I20260808 13:04:21.224026 14250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4c85a6e0a3a40798885880c1fea77e0" candidate_uuid: "5daa346ccf3c406c9acda65c08d4fab6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebee817e0f46434d89c19aea5e53fa04"
I20260808 13:04:21.224200 13767 raft_consensus.cc:3037] T e4c85a6e0a3a40798885880c1fea77e0 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:21.224304 14250 raft_consensus.cc:3037] T e4c85a6e0a3a40798885880c1fea77e0 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:21.227274 13767 raft_consensus.cc:2454] T e4c85a6e0a3a40798885880c1fea77e0 P acf697814b5e4390ae1e3c5faf1a4d62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5daa346ccf3c406c9acda65c08d4fab6 in term 1.
I20260808 13:04:21.227294 14250 raft_consensus.cc:2454] T e4c85a6e0a3a40798885880c1fea77e0 P ebee817e0f46434d89c19aea5e53fa04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5daa346ccf3c406c9acda65c08d4fab6 in term 1.
I20260808 13:04:21.227706 13892 leader_election.cc:302] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5daa346ccf3c406c9acda65c08d4fab6, acf697814b5e4390ae1e3c5faf1a4d62; no voters:
I20260808 13:04:21.228199 14390 raft_consensus.cc:2781] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:04:21.228663 14390 raft_consensus.cc:686] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6 [term 1 LEADER]: Becoming Leader. State: Replica: 5daa346ccf3c406c9acda65c08d4fab6, State: Running, Role: LEADER
I20260808 13:04:21.229130 14390 consensus_queue.cc:237] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:21.232661 13513 catalog_manager.cc:5495] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5daa346ccf3c406c9acda65c08d4fab6 (127.1.124.130). New cstate: current_term: 1 leader_uuid: "5daa346ccf3c406c9acda65c08d4fab6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } health_report { overall_health: UNKNOWN } } }
Aug 08 13:04:21 dist-test-slave-t3q3 krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194257, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.190@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM
I20260808 13:04:21.340699 14444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:21.322494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:36029 (local address 127.0.0.1:49008)
0808 13:04:21.323334 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:21.323349 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:21.323522 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:21.323862 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:21.323866 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:21.324713 (+ 847us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:21.324942 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:21.324953 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.325999 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.326006 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:21.326966 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:21.326977 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.327696 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.327702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:21.327897 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:21.331568 (+ 3671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:21.331587 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:21.334398 (+ 2811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:21.338132 (+ 3734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:21.338144 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:21.338150 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:21.338634 (+ 484us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:21.338980 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:21.338984 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:21.338986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:21.339134 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:21.339624 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:21.339630 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:21.339797 (+ 167us) client_negotiation.cc:769] Sending connection context
0808 13:04:21.339947 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 13:04:21.340312 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":722,"thread_start_us":568,"threads_started":1}
I20260808 13:04:21.340837 14443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:21.322511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49008 (local address 127.1.124.190:36029)
0808 13:04:21.323191 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:21.323199 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:21.323437 (+ 238us) server_negotiation.cc:407] Connection header received
0808 13:04:21.323659 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:21.323662 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:21.323724 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:21.323835 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:21.325107 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.325865 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.327105 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.327525 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.327678 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:21.334680 (+ 7002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:21.334689 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:21.334696 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:21.334734 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:21.337974 (+ 3240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:21.338752 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:21.338756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:21.338758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:21.338876 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:21.339234 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:21.339237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:21.339239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:21.339524 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:21.339648 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:21.340031 (+ 383us) server_negotiation.cc:299] Negotiation successful
0808 13:04:21.340464 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":163,"server-negotiator.queue_time_us":499,"thread_start_us":329,"threads_started":1}
I20260808 13:04:21.357088 13513 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:49008:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 13:04:21.358268 13513 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:04:21.366101 13734 tablet_service.cc:1467] Processing CreateTablet for tablet 0176a78cac654650a2505954d28dc636 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8d650661a48245a889273fe6be93b2a0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 13:04:21.366726 13734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0176a78cac654650a2505954d28dc636. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:21.366760 13978 tablet_service.cc:1467] Processing CreateTablet for tablet 0176a78cac654650a2505954d28dc636 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8d650661a48245a889273fe6be93b2a0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 13:04:21.366833 14221 tablet_service.cc:1467] Processing CreateTablet for tablet 0176a78cac654650a2505954d28dc636 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=8d650661a48245a889273fe6be93b2a0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 13:04:21.367300 13978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0176a78cac654650a2505954d28dc636. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:21.367475 14221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0176a78cac654650a2505954d28dc636. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:21.377408 14450 tablet_bootstrap.cc:492] T 0176a78cac654650a2505954d28dc636 P ebee817e0f46434d89c19aea5e53fa04: Bootstrap starting.
I20260808 13:04:21.380107 14449 tablet_bootstrap.cc:492] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62: Bootstrap starting.
I20260808 13:04:21.382627 14449 tablet_bootstrap.cc:654] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:21.384040 14450 tablet_bootstrap.cc:654] T 0176a78cac654650a2505954d28dc636 P ebee817e0f46434d89c19aea5e53fa04: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:21.384593 14445 tablet_bootstrap.cc:492] T 0176a78cac654650a2505954d28dc636 P 5daa346ccf3c406c9acda65c08d4fab6: Bootstrap starting.
I20260808 13:04:21.386447 14445 tablet_bootstrap.cc:654] T 0176a78cac654650a2505954d28dc636 P 5daa346ccf3c406c9acda65c08d4fab6: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:21.387027 14450 tablet_bootstrap.cc:492] T 0176a78cac654650a2505954d28dc636 P ebee817e0f46434d89c19aea5e53fa04: No bootstrap required, opened a new log
I20260808 13:04:21.387076 14449 tablet_bootstrap.cc:492] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62: No bootstrap required, opened a new log
I20260808 13:04:21.387197 14450 ts_tablet_manager.cc:1397] T 0176a78cac654650a2505954d28dc636 P ebee817e0f46434d89c19aea5e53fa04: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.002s
I20260808 13:04:21.387244 14449 ts_tablet_manager.cc:1397] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.004s
I20260808 13:04:21.388015 14449 raft_consensus.cc:348] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:21.388377 14449 raft_consensus.cc:374] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:21.388212 14450 raft_consensus.cc:348] T 0176a78cac654650a2505954d28dc636 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:21.388490 14449 raft_consensus.cc:729] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf697814b5e4390ae1e3c5faf1a4d62, State: Initialized, Role: FOLLOWER
I20260808 13:04:21.388533 14450 raft_consensus.cc:374] T 0176a78cac654650a2505954d28dc636 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:21.388665 14450 raft_consensus.cc:729] T 0176a78cac654650a2505954d28dc636 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebee817e0f46434d89c19aea5e53fa04, State: Initialized, Role: FOLLOWER
I20260808 13:04:21.388795 14449 consensus_queue.cc:260] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:21.388998 14450 consensus_queue.cc:260] T 0176a78cac654650a2505954d28dc636 P ebee817e0f46434d89c19aea5e53fa04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:21.393472 14445 tablet_bootstrap.cc:492] T 0176a78cac654650a2505954d28dc636 P 5daa346ccf3c406c9acda65c08d4fab6: No bootstrap required, opened a new log
I20260808 13:04:21.393764 14445 ts_tablet_manager.cc:1397] T 0176a78cac654650a2505954d28dc636 P 5daa346ccf3c406c9acda65c08d4fab6: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 13:04:21.393890 14450 ts_tablet_manager.cc:1428] T 0176a78cac654650a2505954d28dc636 P ebee817e0f46434d89c19aea5e53fa04: Time spent starting tablet: real 0.007s user 0.001s sys 0.005s
I20260808 13:04:21.394891 14445 raft_consensus.cc:348] T 0176a78cac654650a2505954d28dc636 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:21.395306 14445 raft_consensus.cc:374] T 0176a78cac654650a2505954d28dc636 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:21.395478 14445 raft_consensus.cc:729] T 0176a78cac654650a2505954d28dc636 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5daa346ccf3c406c9acda65c08d4fab6, State: Initialized, Role: FOLLOWER
I20260808 13:04:21.395874 14445 consensus_queue.cc:260] T 0176a78cac654650a2505954d28dc636 P 5daa346ccf3c406c9acda65c08d4fab6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:21.396512 14390 tablet_replica.cc:439] T 0176a78cac654650a2505954d28dc636 P 5daa346ccf3c406c9acda65c08d4fab6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } } }
I20260808 13:04:21.396796 14390 tablet_replica.cc:442] T 0176a78cac654650a2505954d28dc636 P 5daa346ccf3c406c9acda65c08d4fab6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:04:21.397383 14445 ts_tablet_manager.cc:1428] T 0176a78cac654650a2505954d28dc636 P 5daa346ccf3c406c9acda65c08d4fab6: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 13:04:21.399758 14449 ts_tablet_manager.cc:1428] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62: Time spent starting tablet: real 0.012s user 0.004s sys 0.008s
I20260808 13:04:21.400677 14455 tablet_replica.cc:439] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } } }
I20260808 13:04:21.401059 14455 tablet_replica.cc:442] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:04:21.400090 14456 tablet_replica.cc:439] T 0176a78cac654650a2505954d28dc636 P ebee817e0f46434d89c19aea5e53fa04: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } } }
I20260808 13:04:21.402309 14456 tablet_replica.cc:442] T 0176a78cac654650a2505954d28dc636 P ebee817e0f46434d89c19aea5e53fa04: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:04:21.577646 14455 raft_consensus.cc:482] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:04:21.577942 14455 raft_consensus.cc:504] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:21.578889 14455 leader_election.cc:288] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5daa346ccf3c406c9acda65c08d4fab6 (127.1.124.130:40305), ebee817e0f46434d89c19aea5e53fa04 (127.1.124.131:39001)
I20260808 13:04:21.579494 14014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0176a78cac654650a2505954d28dc636" candidate_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5daa346ccf3c406c9acda65c08d4fab6" is_pre_election: true
I20260808 13:04:21.579505 14250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0176a78cac654650a2505954d28dc636" candidate_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebee817e0f46434d89c19aea5e53fa04" is_pre_election: true
I20260808 13:04:21.579785 14014 raft_consensus.cc:2454] T 0176a78cac654650a2505954d28dc636 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf697814b5e4390ae1e3c5faf1a4d62 in term 0.
I20260808 13:04:21.579818 14250 raft_consensus.cc:2454] T 0176a78cac654650a2505954d28dc636 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf697814b5e4390ae1e3c5faf1a4d62 in term 0.
I20260808 13:04:21.580360 13647 leader_election.cc:302] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5daa346ccf3c406c9acda65c08d4fab6, acf697814b5e4390ae1e3c5faf1a4d62; no voters:
I20260808 13:04:21.580680 14455 raft_consensus.cc:2781] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:04:21.580821 14455 raft_consensus.cc:482] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:04:21.580927 14455 raft_consensus.cc:3037] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:21.582998 14455 raft_consensus.cc:504] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:21.583690 14455 leader_election.cc:288] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62 [CANDIDATE]: Term 1 election: Requested vote from peers 5daa346ccf3c406c9acda65c08d4fab6 (127.1.124.130:40305), ebee817e0f46434d89c19aea5e53fa04 (127.1.124.131:39001)
I20260808 13:04:21.584311 14250 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0176a78cac654650a2505954d28dc636" candidate_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebee817e0f46434d89c19aea5e53fa04"
I20260808 13:04:21.584544 14250 raft_consensus.cc:3037] T 0176a78cac654650a2505954d28dc636 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:21.584491 14014 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0176a78cac654650a2505954d28dc636" candidate_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5daa346ccf3c406c9acda65c08d4fab6"
I20260808 13:04:21.584743 14014 raft_consensus.cc:3037] T 0176a78cac654650a2505954d28dc636 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:21.586948 14250 raft_consensus.cc:2454] T 0176a78cac654650a2505954d28dc636 P ebee817e0f46434d89c19aea5e53fa04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf697814b5e4390ae1e3c5faf1a4d62 in term 1.
I20260808 13:04:21.587467 13648 leader_election.cc:302] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: acf697814b5e4390ae1e3c5faf1a4d62, ebee817e0f46434d89c19aea5e53fa04; no voters:
I20260808 13:04:21.587782 14455 raft_consensus.cc:2781] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:04:21.588353 14455 raft_consensus.cc:686] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62 [term 1 LEADER]: Becoming Leader. State: Replica: acf697814b5e4390ae1e3c5faf1a4d62, State: Running, Role: LEADER
I20260808 13:04:21.588636 14014 raft_consensus.cc:2454] T 0176a78cac654650a2505954d28dc636 P 5daa346ccf3c406c9acda65c08d4fab6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf697814b5e4390ae1e3c5faf1a4d62 in term 1.
I20260808 13:04:21.589025 14455 consensus_queue.cc:237] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:21.590888 14458 tablet_replica.cc:439] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62: TxnStatusTablet state changed. Reason: New leader acf697814b5e4390ae1e3c5faf1a4d62. Latest consensus state: current_term: 1 leader_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } } }
I20260808 13:04:21.591266 14458 tablet_replica.cc:442] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62: This TxnStatusTablet replica's current role is: LEADER
I20260808 13:04:21.592118 14460 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 13:04:21.592404 14460 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 13:04:21.592864 13511 catalog_manager.cc:5495] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf697814b5e4390ae1e3c5faf1a4d62 (127.1.124.129). New cstate: current_term: 1 leader_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } health_report { overall_health: UNKNOWN } } }
I20260808 13:04:21.630391 14427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:21.622399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57330 (local address 127.1.124.129:34901)
0808 13:04:21.622652 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:21.622661 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:21.623045 (+ 384us) server_negotiation.cc:407] Connection header received
0808 13:04:21.623351 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:21.623357 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:21.623459 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:21.623573 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:04:21.624106 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.625434 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.627851 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.628641 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.628785 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:21.629479 (+ 694us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:04:21.629658 (+ 179us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:04:21.629765 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:21.629955 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 13:04:21.630058 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 13:04:21.630957 14444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:21.622450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:34901 (local address 127.0.0.1:57330)
0808 13:04:21.622917 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:21.622939 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:21.623131 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:21.623621 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:21.623629 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:21.623648 (+ 19us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:04:21.623907 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:21.623921 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.625626 (+ 1705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.625634 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:21.627657 (+ 2023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:21.627678 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.628816 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.628824 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:21.629137 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:21.629239 (+ 102us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:04:21.629780 (+ 541us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:04:21.629807 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 13:04:21.629904 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 13:04:21.630127 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251}
I20260808 13:04:21.636499 14250 raft_consensus.cc:1261] T 0176a78cac654650a2505954d28dc636 P ebee817e0f46434d89c19aea5e53fa04 [term 1 FOLLOWER]: Refusing update from remote peer acf697814b5e4390ae1e3c5faf1a4d62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:04:21.636611 14014 raft_consensus.cc:1261] T 0176a78cac654650a2505954d28dc636 P 5daa346ccf3c406c9acda65c08d4fab6 [term 1 FOLLOWER]: Refusing update from remote peer acf697814b5e4390ae1e3c5faf1a4d62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:04:21.637317 14458 consensus_queue.cc:1035] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:04:21.637724 14455 consensus_queue.cc:1035] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:04:21.655426 14456 tablet_replica.cc:439] T 0176a78cac654650a2505954d28dc636 P ebee817e0f46434d89c19aea5e53fa04: TxnStatusTablet state changed. Reason: New leader acf697814b5e4390ae1e3c5faf1a4d62. Latest consensus state: current_term: 1 leader_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } } }
I20260808 13:04:21.655776 14456 tablet_replica.cc:442] T 0176a78cac654650a2505954d28dc636 P ebee817e0f46434d89c19aea5e53fa04: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:04:21.660346 14458 tablet_replica.cc:439] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } } }
I20260808 13:04:21.660622 14458 tablet_replica.cc:442] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62: This TxnStatusTablet replica's current role is: LEADER
I20260808 13:04:21.662925 14466 mvcc.cc:205] Tried to move back new op lower bound from 7316251695655772160 to 7316251695471321088. Current Snapshot: MvccSnapshot[applied={T|T < 7316251695655772160 or (T in {7316251695655772160})}]
I20260808 13:04:21.674036 14390 tablet_replica.cc:439] T 0176a78cac654650a2505954d28dc636 P 5daa346ccf3c406c9acda65c08d4fab6: TxnStatusTablet state changed. Reason: New leader acf697814b5e4390ae1e3c5faf1a4d62. Latest consensus state: current_term: 1 leader_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } } }
I20260808 13:04:21.674484 14390 tablet_replica.cc:442] T 0176a78cac654650a2505954d28dc636 P 5daa346ccf3c406c9acda65c08d4fab6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:04:21.679028 14455 tablet_replica.cc:439] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } } }
I20260808 13:04:21.679406 14455 tablet_replica.cc:442] T 0176a78cac654650a2505954d28dc636 P acf697814b5e4390ae1e3c5faf1a4d62: This TxnStatusTablet replica's current role is: LEADER
I20260808 13:04:21.681342 14470 mvcc.cc:205] Tried to move back new op lower bound from 7316251695655772160 to 7316251695471321088. Current Snapshot: MvccSnapshot[applied={T|T < 7316251695655772160}]
I20260808 13:04:21.688794 14456 tablet_replica.cc:439] T 0176a78cac654650a2505954d28dc636 P ebee817e0f46434d89c19aea5e53fa04: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } } }
I20260808 13:04:21.689137 14456 tablet_replica.cc:442] T 0176a78cac654650a2505954d28dc636 P ebee817e0f46434d89c19aea5e53fa04: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:04:21.694377 14390 tablet_replica.cc:439] T 0176a78cac654650a2505954d28dc636 P 5daa346ccf3c406c9acda65c08d4fab6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } } }
I20260808 13:04:21.694734 14390 tablet_replica.cc:442] T 0176a78cac654650a2505954d28dc636 P 5daa346ccf3c406c9acda65c08d4fab6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 13:04:21.720172 14488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:21.699413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51402 (local address 127.1.124.130:40305)
0808 13:04:21.712589 (+ 13176us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:21.712598 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:21.713240 (+ 642us) server_negotiation.cc:407] Connection header received
0808 13:04:21.713506 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:21.713513 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:21.713621 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:21.713788 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:04:21.714621 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.715860 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.717521 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.718117 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.718267 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:21.718903 (+ 636us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:04:21.719475 (+ 572us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:04:21.719649 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:21.719738 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 13:04:21.719820 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12980,"thread_start_us":16674,"threads_started":1}
I20260808 13:04:21.736159 14427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:21.730074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:55391 (local address 127.1.124.129:34901)
0808 13:04:21.730314 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:21.730324 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:21.730841 (+ 517us) server_negotiation.cc:407] Connection header received
0808 13:04:21.731062 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:21.731067 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:21.731161 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:21.731263 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:04:21.731778 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.732715 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.734266 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.734705 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.734873 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:21.735324 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:04:21.735444 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:04:21.735542 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:21.735709 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 13:04:21.735787 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 13:04:21.736526 14492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:21.729940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:34901 (local address 127.1.124.130:55391)
0808 13:04:21.730734 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:21.730750 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:21.730901 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:21.731289 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:21.731293 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:21.731304 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:04:21.731602 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:21.731611 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.732858 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.732864 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:21.734118 (+ 1254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:21.734131 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.734889 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.734895 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:21.735079 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:21.735132 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:04:21.735558 (+ 426us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:04:21.735571 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 13:04:21.735674 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 13:04:21.735810 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":516,"threads_started":1}
I20260808 13:04:21.745641 14390 consensus_queue.cc:1035] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:04:21.750988 14249 raft_consensus.cc:1261] T e4c85a6e0a3a40798885880c1fea77e0 P ebee817e0f46434d89c19aea5e53fa04 [term 1 FOLLOWER]: Refusing update from remote peer 5daa346ccf3c406c9acda65c08d4fab6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:04:21.763440 14498 consensus_queue.cc:1035] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 13:04:21.804776 13896 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.1.124.131:39001, user_credentials={real_user=slave}} blocked reactor thread for 53129.4us
I20260808 13:04:21.835786 14488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:21.815866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:52463 (local address 127.1.124.130:40305)
0808 13:04:21.816045 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:21.816052 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:21.820413 (+ 4361us) server_negotiation.cc:407] Connection header received
0808 13:04:21.820973 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:21.820981 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:21.821098 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:21.821177 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:04:21.825441 (+ 4264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.826554 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.832630 (+ 6076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.833315 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.833420 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:21.835053 (+ 1633us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:04:21.835161 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:04:21.835268 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:21.835397 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 13:04:21.835480 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 13:04:21.836287 14504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:21.815500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:40305 (local address 127.1.124.129:52463)
0808 13:04:21.820189 (+ 4689us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:21.820254 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:21.820696 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:21.824191 (+ 3495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:21.824229 (+ 38us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:21.824250 (+ 21us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:04:21.825083 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:21.825131 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.830379 (+ 5248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.830387 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:21.832432 (+ 2045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:21.832448 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.833729 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.833737 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:21.834600 (+ 863us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:21.834705 (+ 105us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:04:21.835285 (+ 580us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:04:21.835298 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 13:04:21.835365 (+ 67us) client_negotiation.cc:240] Negotiation successful
0808 13:04:21.835523 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4173,"thread_start_us":420,"threads_started":1}
I20260808 13:04:21.889799 13511 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:04:21.890107 13511 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:48996:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:04:21.893764 13511 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44: Sending DeleteTablet for 3 replicas of tablet e4c85a6e0a3a40798885880c1fea77e0
I20260808 13:04:21.894690 13734 tablet_service.cc:1514] Processing DeleteTablet for tablet e4c85a6e0a3a40798885880c1fea77e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:04:21 UTC) from {username='oryx', principal='oryx/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:57314
I20260808 13:04:21.895030 14221 tablet_service.cc:1514] Processing DeleteTablet for tablet e4c85a6e0a3a40798885880c1fea77e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:04:21 UTC) from {username='oryx', principal='oryx/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:50200
I20260808 13:04:21.896031 13978 tablet_service.cc:1514] Processing DeleteTablet for tablet e4c85a6e0a3a40798885880c1fea77e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:04:21 UTC) from {username='oryx', principal='oryx/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:51392
I20260808 13:04:21.912498 14517 tablet_replica.cc:330] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6: stopping tablet replica
I20260808 13:04:21.912804 14517 raft_consensus.cc:2229] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:04:21.912930 14516 tablet_replica.cc:330] T e4c85a6e0a3a40798885880c1fea77e0 P ebee817e0f46434d89c19aea5e53fa04: stopping tablet replica
I20260808 13:04:21.913208 14516 raft_consensus.cc:2229] T e4c85a6e0a3a40798885880c1fea77e0 P ebee817e0f46434d89c19aea5e53fa04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:04:21.913465 14516 raft_consensus.cc:2258] T e4c85a6e0a3a40798885880c1fea77e0 P ebee817e0f46434d89c19aea5e53fa04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:04:21.913520 14517 raft_consensus.cc:2258] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:04:21.915128 14517 ts_tablet_manager.cc:1905] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:21.923187 14512 tablet_replica.cc:330] T e4c85a6e0a3a40798885880c1fea77e0 P acf697814b5e4390ae1e3c5faf1a4d62: stopping tablet replica
I20260808 13:04:21.923581 14512 raft_consensus.cc:2229] T e4c85a6e0a3a40798885880c1fea77e0 P acf697814b5e4390ae1e3c5faf1a4d62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:04:21.923933 14512 raft_consensus.cc:2258] T e4c85a6e0a3a40798885880c1fea77e0 P acf697814b5e4390ae1e3c5faf1a4d62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:04:21.924587 14517 ts_tablet_manager.cc:1918] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:04:21.924736 14517 log.cc:1196] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/e4c85a6e0a3a40798885880c1fea77e0
I20260808 13:04:21.924945 14516 ts_tablet_manager.cc:1905] T e4c85a6e0a3a40798885880c1fea77e0 P ebee817e0f46434d89c19aea5e53fa04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:21.925225 14517 ts_tablet_manager.cc:1939] T e4c85a6e0a3a40798885880c1fea77e0 P 5daa346ccf3c406c9acda65c08d4fab6: Deleting consensus metadata
I20260808 13:04:21.925952 14512 ts_tablet_manager.cc:1905] T e4c85a6e0a3a40798885880c1fea77e0 P acf697814b5e4390ae1e3c5faf1a4d62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:21.926540 13492 catalog_manager.cc:4841] TS 5daa346ccf3c406c9acda65c08d4fab6 (127.1.124.130:40305): tablet e4c85a6e0a3a40798885880c1fea77e0 (table test-table [id=85c69886fd4843818e0f2b621befd594]) successfully deleted
I20260808 13:04:21.931775 14512 ts_tablet_manager.cc:1918] T e4c85a6e0a3a40798885880c1fea77e0 P acf697814b5e4390ae1e3c5faf1a4d62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:04:21.931967 14512 log.cc:1196] T e4c85a6e0a3a40798885880c1fea77e0 P acf697814b5e4390ae1e3c5faf1a4d62: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/e4c85a6e0a3a40798885880c1fea77e0
I20260808 13:04:21.932281 14516 ts_tablet_manager.cc:1918] T e4c85a6e0a3a40798885880c1fea77e0 P ebee817e0f46434d89c19aea5e53fa04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:04:21.932447 14516 log.cc:1196] T e4c85a6e0a3a40798885880c1fea77e0 P ebee817e0f46434d89c19aea5e53fa04: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/e4c85a6e0a3a40798885880c1fea77e0
I20260808 13:04:21.932561 14512 ts_tablet_manager.cc:1939] T e4c85a6e0a3a40798885880c1fea77e0 P acf697814b5e4390ae1e3c5faf1a4d62: Deleting consensus metadata
I20260808 13:04:21.932950 14516 ts_tablet_manager.cc:1939] T e4c85a6e0a3a40798885880c1fea77e0 P ebee817e0f46434d89c19aea5e53fa04: Deleting consensus metadata
I20260808 13:04:21.934096 13489 catalog_manager.cc:4841] TS acf697814b5e4390ae1e3c5faf1a4d62 (127.1.124.129:34901): tablet e4c85a6e0a3a40798885880c1fea77e0 (table test-table [id=85c69886fd4843818e0f2b621befd594]) successfully deleted
I20260808 13:04:21.934223 13493 catalog_manager.cc:4841] TS ebee817e0f46434d89c19aea5e53fa04 (127.1.124.131:39001): tablet e4c85a6e0a3a40798885880c1fea77e0 (table test-table [id=85c69886fd4843818e0f2b621befd594]) successfully deleted
Aug 08 13:04:21 dist-test-slave-t3q3 krb5kdc[13438](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 13:04:21 dist-test-slave-t3q3 krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:21 dist-test-slave-t3q3 krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 13:04:22.002931 14528 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:21.975648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49010 (local address 127.1.124.190:36029)
0808 13:04:21.991104 (+ 15456us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:21.991111 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:21.991262 (+ 151us) server_negotiation.cc:407] Connection header received
0808 13:04:21.991547 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:21.991553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:21.991643 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:21.991800 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:21.993517 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.994312 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.995708 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:21.996231 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:21.996380 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:22.002612 (+ 6232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49010: BlockingRecv error: recv got EOF from 127.0.0.1:49010 (error 108)
Metrics: {"server-negotiator.queue_time_us":15295,"thread_start_us":15483,"threads_started":1}
Aug 08 13:04:22 dist-test-slave-t3q3 krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM
I20260808 13:04:22.021610 14528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:22.006903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49026 (local address 127.1.124.190:36029)
0808 13:04:22.007082 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:22.007087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:22.007108 (+ 21us) server_negotiation.cc:407] Connection header received
0808 13:04:22.007278 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:22.007286 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:22.007371 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:22.007433 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:22.008861 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.009624 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.010891 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.011320 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.011464 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:22.015480 (+ 4016us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:22.015485 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:22.015488 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:22.015523 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:22.018726 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:22.019426 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:22.019433 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:22.019436 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:22.019590 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:22.020023 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:22.020029 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:22.020032 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:22.020384 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:22.020666 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:22.020973 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 13:04:22.021225 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 13:04:22.025504 13511 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:04:22.026155 13511 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:04:22.033638 13734 tablet_service.cc:1467] Processing CreateTablet for tablet 901395a191464e99afa502f79fa98e39 (DEFAULT_TABLE table=test-table [id=0c451e6f9df24292badc4f43f20c743e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:04:22.033766 14221 tablet_service.cc:1467] Processing CreateTablet for tablet 901395a191464e99afa502f79fa98e39 (DEFAULT_TABLE table=test-table [id=0c451e6f9df24292badc4f43f20c743e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:04:22.034204 13734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 901395a191464e99afa502f79fa98e39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:22.034173 14221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 901395a191464e99afa502f79fa98e39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:22.040688 13978 tablet_service.cc:1467] Processing CreateTablet for tablet 901395a191464e99afa502f79fa98e39 (DEFAULT_TABLE table=test-table [id=0c451e6f9df24292badc4f43f20c743e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:04:22.041200 13978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 901395a191464e99afa502f79fa98e39. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:22.052903 14546 tablet_bootstrap.cc:492] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62: Bootstrap starting.
I20260808 13:04:22.055660 14546 tablet_bootstrap.cc:654] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:22.057466 14544 tablet_bootstrap.cc:492] T 901395a191464e99afa502f79fa98e39 P ebee817e0f46434d89c19aea5e53fa04: Bootstrap starting.
I20260808 13:04:22.059226 14544 tablet_bootstrap.cc:654] T 901395a191464e99afa502f79fa98e39 P ebee817e0f46434d89c19aea5e53fa04: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:22.061992 14544 tablet_bootstrap.cc:492] T 901395a191464e99afa502f79fa98e39 P ebee817e0f46434d89c19aea5e53fa04: No bootstrap required, opened a new log
I20260808 13:04:22.062173 14544 ts_tablet_manager.cc:1397] T 901395a191464e99afa502f79fa98e39 P ebee817e0f46434d89c19aea5e53fa04: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 13:04:22.062825 14544 raft_consensus.cc:348] T 901395a191464e99afa502f79fa98e39 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:22.063058 14544 raft_consensus.cc:374] T 901395a191464e99afa502f79fa98e39 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:22.063151 14544 raft_consensus.cc:729] T 901395a191464e99afa502f79fa98e39 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebee817e0f46434d89c19aea5e53fa04, State: Initialized, Role: FOLLOWER
I20260808 13:04:22.063433 14544 consensus_queue.cc:260] T 901395a191464e99afa502f79fa98e39 P ebee817e0f46434d89c19aea5e53fa04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:22.066766 14546 tablet_bootstrap.cc:492] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62: No bootstrap required, opened a new log
I20260808 13:04:22.066957 14546 ts_tablet_manager.cc:1397] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62: Time spent bootstrapping tablet: real 0.014s user 0.002s sys 0.003s
I20260808 13:04:22.067664 14546 raft_consensus.cc:348] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:22.067922 14546 raft_consensus.cc:374] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:22.068094 14546 raft_consensus.cc:729] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf697814b5e4390ae1e3c5faf1a4d62, State: Initialized, Role: FOLLOWER
I20260808 13:04:22.068414 14546 consensus_queue.cc:260] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:22.069016 14546 ts_tablet_manager.cc:1428] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 13:04:22.069442 14544 ts_tablet_manager.cc:1428] T 901395a191464e99afa502f79fa98e39 P ebee817e0f46434d89c19aea5e53fa04: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260808 13:04:22.072628 14548 tablet_bootstrap.cc:492] T 901395a191464e99afa502f79fa98e39 P 5daa346ccf3c406c9acda65c08d4fab6: Bootstrap starting.
I20260808 13:04:22.074690 14548 tablet_bootstrap.cc:654] T 901395a191464e99afa502f79fa98e39 P 5daa346ccf3c406c9acda65c08d4fab6: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:22.077077 14548 tablet_bootstrap.cc:492] T 901395a191464e99afa502f79fa98e39 P 5daa346ccf3c406c9acda65c08d4fab6: No bootstrap required, opened a new log
I20260808 13:04:22.077237 14548 ts_tablet_manager.cc:1397] T 901395a191464e99afa502f79fa98e39 P 5daa346ccf3c406c9acda65c08d4fab6: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 13:04:22.078070 14548 raft_consensus.cc:348] T 901395a191464e99afa502f79fa98e39 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:22.078343 14548 raft_consensus.cc:374] T 901395a191464e99afa502f79fa98e39 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:22.078454 14548 raft_consensus.cc:729] T 901395a191464e99afa502f79fa98e39 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5daa346ccf3c406c9acda65c08d4fab6, State: Initialized, Role: FOLLOWER
I20260808 13:04:22.078711 14548 consensus_queue.cc:260] T 901395a191464e99afa502f79fa98e39 P 5daa346ccf3c406c9acda65c08d4fab6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:22.079291 14548 ts_tablet_manager.cc:1428] T 901395a191464e99afa502f79fa98e39 P 5daa346ccf3c406c9acda65c08d4fab6: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260808 13:04:22.458040 14458 raft_consensus.cc:482] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:04:22.458308 14458 raft_consensus.cc:504] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:22.458966 14458 leader_election.cc:288] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5daa346ccf3c406c9acda65c08d4fab6 (127.1.124.130:40305), ebee817e0f46434d89c19aea5e53fa04 (127.1.124.131:39001)
I20260808 13:04:22.459586 14012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "901395a191464e99afa502f79fa98e39" candidate_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5daa346ccf3c406c9acda65c08d4fab6" is_pre_election: true
I20260808 13:04:22.459610 14249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "901395a191464e99afa502f79fa98e39" candidate_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebee817e0f46434d89c19aea5e53fa04" is_pre_election: true
I20260808 13:04:22.459826 14012 raft_consensus.cc:2454] T 901395a191464e99afa502f79fa98e39 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf697814b5e4390ae1e3c5faf1a4d62 in term 0.
I20260808 13:04:22.459924 14249 raft_consensus.cc:2454] T 901395a191464e99afa502f79fa98e39 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate acf697814b5e4390ae1e3c5faf1a4d62 in term 0.
I20260808 13:04:22.460389 13647 leader_election.cc:302] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5daa346ccf3c406c9acda65c08d4fab6, acf697814b5e4390ae1e3c5faf1a4d62; no voters:
I20260808 13:04:22.460706 14458 raft_consensus.cc:2781] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:04:22.460796 14458 raft_consensus.cc:482] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:04:22.460875 14458 raft_consensus.cc:3037] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:22.462816 14458 raft_consensus.cc:504] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:22.463361 14458 leader_election.cc:288] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62 [CANDIDATE]: Term 1 election: Requested vote from peers 5daa346ccf3c406c9acda65c08d4fab6 (127.1.124.130:40305), ebee817e0f46434d89c19aea5e53fa04 (127.1.124.131:39001)
I20260808 13:04:22.463837 14249 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "901395a191464e99afa502f79fa98e39" candidate_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ebee817e0f46434d89c19aea5e53fa04"
I20260808 13:04:22.463845 14012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "901395a191464e99afa502f79fa98e39" candidate_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5daa346ccf3c406c9acda65c08d4fab6"
I20260808 13:04:22.464047 14012 raft_consensus.cc:3037] T 901395a191464e99afa502f79fa98e39 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:22.464052 14249 raft_consensus.cc:3037] T 901395a191464e99afa502f79fa98e39 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:22.466234 14249 raft_consensus.cc:2454] T 901395a191464e99afa502f79fa98e39 P ebee817e0f46434d89c19aea5e53fa04 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf697814b5e4390ae1e3c5faf1a4d62 in term 1.
I20260808 13:04:22.466284 14012 raft_consensus.cc:2454] T 901395a191464e99afa502f79fa98e39 P 5daa346ccf3c406c9acda65c08d4fab6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate acf697814b5e4390ae1e3c5faf1a4d62 in term 1.
I20260808 13:04:22.466684 13648 leader_election.cc:302] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: acf697814b5e4390ae1e3c5faf1a4d62, ebee817e0f46434d89c19aea5e53fa04; no voters:
I20260808 13:04:22.466970 14458 raft_consensus.cc:2781] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:04:22.467350 14458 raft_consensus.cc:686] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62 [term 1 LEADER]: Becoming Leader. State: Replica: acf697814b5e4390ae1e3c5faf1a4d62, State: Running, Role: LEADER
I20260808 13:04:22.467608 14458 consensus_queue.cc:237] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:22.470342 13511 catalog_manager.cc:5495] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62 reported cstate change: term changed from 0 to 1, leader changed from <none> to acf697814b5e4390ae1e3c5faf1a4d62 (127.1.124.129). New cstate: current_term: 1 leader_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } health_report { overall_health: UNKNOWN } } }
I20260808 13:04:22.517953 14555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:22.513260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57332 (local address 127.1.124.129:34901)
0808 13:04:22.513715 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:22.513721 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:22.513738 (+ 17us) server_negotiation.cc:407] Connection header received
0808 13:04:22.513798 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:22.513801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:22.513867 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:22.513953 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:04:22.514299 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.515044 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.516319 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.516738 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.516845 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:22.517323 (+ 478us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:04:22.517432 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:04:22.517501 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:22.517637 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 13:04:22.517700 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":203,"threads_started":1}
I20260808 13:04:22.520941 14249 raft_consensus.cc:1261] T 901395a191464e99afa502f79fa98e39 P ebee817e0f46434d89c19aea5e53fa04 [term 1 FOLLOWER]: Refusing update from remote peer acf697814b5e4390ae1e3c5faf1a4d62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:04:22.520962 14012 raft_consensus.cc:1261] T 901395a191464e99afa502f79fa98e39 P 5daa346ccf3c406c9acda65c08d4fab6 [term 1 FOLLOWER]: Refusing update from remote peer acf697814b5e4390ae1e3c5faf1a4d62: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:04:22.521600 14458 consensus_queue.cc:1035] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:04:22.521911 14552 consensus_queue.cc:1035] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:04:22.534703 13513 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:04:22.534940 13513 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:04:22.537833 13513 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44: Sending DeleteTablet for 3 replicas of tablet 901395a191464e99afa502f79fa98e39
I20260808 13:04:22.538602 13734 tablet_service.cc:1514] Processing DeleteTablet for tablet 901395a191464e99afa502f79fa98e39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:04:22 UTC) from {username='oryx', principal='oryx/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:57314
I20260808 13:04:22.538763 13978 tablet_service.cc:1514] Processing DeleteTablet for tablet 901395a191464e99afa502f79fa98e39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:04:22 UTC) from {username='oryx', principal='oryx/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:51392
I20260808 13:04:22.538851 14221 tablet_service.cc:1514] Processing DeleteTablet for tablet 901395a191464e99afa502f79fa98e39 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:04:22 UTC) from {username='oryx', principal='oryx/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:50200
I20260808 13:04:22.539995 13513 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:04:22.540903 13513 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:04:22.543694 14572 tablet_replica.cc:330] T 901395a191464e99afa502f79fa98e39 P ebee817e0f46434d89c19aea5e53fa04: stopping tablet replica
I20260808 13:04:22.543933 14569 tablet_replica.cc:330] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62: stopping tablet replica
I20260808 13:04:22.543915 14572 raft_consensus.cc:2229] T 901395a191464e99afa502f79fa98e39 P ebee817e0f46434d89c19aea5e53fa04 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:04:22.544162 14569 raft_consensus.cc:2229] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:04:22.544246 14572 raft_consensus.cc:2258] T 901395a191464e99afa502f79fa98e39 P ebee817e0f46434d89c19aea5e53fa04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:04:22.544369 14571 tablet_replica.cc:330] T 901395a191464e99afa502f79fa98e39 P 5daa346ccf3c406c9acda65c08d4fab6: stopping tablet replica
I20260808 13:04:22.544637 14571 raft_consensus.cc:2229] T 901395a191464e99afa502f79fa98e39 P 5daa346ccf3c406c9acda65c08d4fab6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:04:22.544916 14571 raft_consensus.cc:2258] T 901395a191464e99afa502f79fa98e39 P 5daa346ccf3c406c9acda65c08d4fab6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:04:22.545652 14569 raft_consensus.cc:2258] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:04:22.546168 14571 ts_tablet_manager.cc:1905] T 901395a191464e99afa502f79fa98e39 P 5daa346ccf3c406c9acda65c08d4fab6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:22.546370 14572 ts_tablet_manager.cc:1905] T 901395a191464e99afa502f79fa98e39 P ebee817e0f46434d89c19aea5e53fa04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:22.549429 13978 tablet_service.cc:1467] Processing CreateTablet for tablet 22d64cc3a2fe4f2fb091c16365dd0407 (DEFAULT_TABLE table=test-table [id=af6b8fa956754956b6c654d849cf6127]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:04:22.549885 13978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22d64cc3a2fe4f2fb091c16365dd0407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:22.550400 14221 tablet_service.cc:1467] Processing CreateTablet for tablet 22d64cc3a2fe4f2fb091c16365dd0407 (DEFAULT_TABLE table=test-table [id=af6b8fa956754956b6c654d849cf6127]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:04:22.550866 14221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22d64cc3a2fe4f2fb091c16365dd0407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:22.553200 14572 ts_tablet_manager.cc:1918] T 901395a191464e99afa502f79fa98e39 P ebee817e0f46434d89c19aea5e53fa04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:04:22.553364 14572 log.cc:1196] T 901395a191464e99afa502f79fa98e39 P ebee817e0f46434d89c19aea5e53fa04: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/901395a191464e99afa502f79fa98e39
I20260808 13:04:22.553612 14571 ts_tablet_manager.cc:1918] T 901395a191464e99afa502f79fa98e39 P 5daa346ccf3c406c9acda65c08d4fab6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:04:22.553810 14571 log.cc:1196] T 901395a191464e99afa502f79fa98e39 P 5daa346ccf3c406c9acda65c08d4fab6: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/901395a191464e99afa502f79fa98e39
I20260808 13:04:22.553874 14572 ts_tablet_manager.cc:1939] T 901395a191464e99afa502f79fa98e39 P ebee817e0f46434d89c19aea5e53fa04: Deleting consensus metadata
I20260808 13:04:22.554420 14571 ts_tablet_manager.cc:1939] T 901395a191464e99afa502f79fa98e39 P 5daa346ccf3c406c9acda65c08d4fab6: Deleting consensus metadata
I20260808 13:04:22.547751 14569 ts_tablet_manager.cc:1905] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:22.549441 13734 tablet_service.cc:1467] Processing CreateTablet for tablet 22d64cc3a2fe4f2fb091c16365dd0407 (DEFAULT_TABLE table=test-table [id=af6b8fa956754956b6c654d849cf6127]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:04:22.554970 13493 catalog_manager.cc:4841] TS ebee817e0f46434d89c19aea5e53fa04 (127.1.124.131:39001): tablet 901395a191464e99afa502f79fa98e39 (table test-table [id=0c451e6f9df24292badc4f43f20c743e]) successfully deleted
I20260808 13:04:22.555358 13734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 22d64cc3a2fe4f2fb091c16365dd0407. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:22.555739 13492 catalog_manager.cc:4841] TS 5daa346ccf3c406c9acda65c08d4fab6 (127.1.124.130:40305): tablet 901395a191464e99afa502f79fa98e39 (table test-table [id=0c451e6f9df24292badc4f43f20c743e]) successfully deleted
I20260808 13:04:22.557781 14548 tablet_bootstrap.cc:492] T 22d64cc3a2fe4f2fb091c16365dd0407 P 5daa346ccf3c406c9acda65c08d4fab6: Bootstrap starting.
I20260808 13:04:22.559396 14548 tablet_bootstrap.cc:654] T 22d64cc3a2fe4f2fb091c16365dd0407 P 5daa346ccf3c406c9acda65c08d4fab6: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:22.559597 14544 tablet_bootstrap.cc:492] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04: Bootstrap starting.
I20260808 13:04:22.561214 14544 tablet_bootstrap.cc:654] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:22.561358 14548 tablet_bootstrap.cc:492] T 22d64cc3a2fe4f2fb091c16365dd0407 P 5daa346ccf3c406c9acda65c08d4fab6: No bootstrap required, opened a new log
I20260808 13:04:22.561534 14548 ts_tablet_manager.cc:1397] T 22d64cc3a2fe4f2fb091c16365dd0407 P 5daa346ccf3c406c9acda65c08d4fab6: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 13:04:22.562178 14548 raft_consensus.cc:348] T 22d64cc3a2fe4f2fb091c16365dd0407 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:22.562456 14548 raft_consensus.cc:374] T 22d64cc3a2fe4f2fb091c16365dd0407 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:22.562556 14548 raft_consensus.cc:729] T 22d64cc3a2fe4f2fb091c16365dd0407 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5daa346ccf3c406c9acda65c08d4fab6, State: Initialized, Role: FOLLOWER
I20260808 13:04:22.562840 14548 consensus_queue.cc:260] T 22d64cc3a2fe4f2fb091c16365dd0407 P 5daa346ccf3c406c9acda65c08d4fab6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:22.563715 14544 tablet_bootstrap.cc:492] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04: No bootstrap required, opened a new log
I20260808 13:04:22.563932 14544 ts_tablet_manager.cc:1397] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260808 13:04:22.564149 14548 ts_tablet_manager.cc:1428] T 22d64cc3a2fe4f2fb091c16365dd0407 P 5daa346ccf3c406c9acda65c08d4fab6: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260808 13:04:22.564956 14569 ts_tablet_manager.cc:1918] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:04:22.564852 14544 raft_consensus.cc:348] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:22.565124 14569 log.cc:1196] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/901395a191464e99afa502f79fa98e39
I20260808 13:04:22.565295 14544 raft_consensus.cc:374] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:22.565426 14544 raft_consensus.cc:729] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ebee817e0f46434d89c19aea5e53fa04, State: Initialized, Role: FOLLOWER
I20260808 13:04:22.565687 14569 ts_tablet_manager.cc:1939] T 901395a191464e99afa502f79fa98e39 P acf697814b5e4390ae1e3c5faf1a4d62: Deleting consensus metadata
I20260808 13:04:22.565706 14544 consensus_queue.cc:260] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:22.566388 14546 tablet_bootstrap.cc:492] T 22d64cc3a2fe4f2fb091c16365dd0407 P acf697814b5e4390ae1e3c5faf1a4d62: Bootstrap starting.
I20260808 13:04:22.566977 14544 ts_tablet_manager.cc:1428] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 13:04:22.567293 13489 catalog_manager.cc:4841] TS acf697814b5e4390ae1e3c5faf1a4d62 (127.1.124.129:34901): tablet 901395a191464e99afa502f79fa98e39 (table test-table [id=0c451e6f9df24292badc4f43f20c743e]) successfully deleted
I20260808 13:04:22.568186 14546 tablet_bootstrap.cc:654] T 22d64cc3a2fe4f2fb091c16365dd0407 P acf697814b5e4390ae1e3c5faf1a4d62: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:22.570393 14546 tablet_bootstrap.cc:492] T 22d64cc3a2fe4f2fb091c16365dd0407 P acf697814b5e4390ae1e3c5faf1a4d62: No bootstrap required, opened a new log
I20260808 13:04:22.570566 14546 ts_tablet_manager.cc:1397] T 22d64cc3a2fe4f2fb091c16365dd0407 P acf697814b5e4390ae1e3c5faf1a4d62: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.002s
I20260808 13:04:22.571301 14546 raft_consensus.cc:348] T 22d64cc3a2fe4f2fb091c16365dd0407 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:22.571489 14546 raft_consensus.cc:374] T 22d64cc3a2fe4f2fb091c16365dd0407 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:22.571552 14546 raft_consensus.cc:729] T 22d64cc3a2fe4f2fb091c16365dd0407 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acf697814b5e4390ae1e3c5faf1a4d62, State: Initialized, Role: FOLLOWER
I20260808 13:04:22.571779 14546 consensus_queue.cc:260] T 22d64cc3a2fe4f2fb091c16365dd0407 P acf697814b5e4390ae1e3c5faf1a4d62 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:22.572264 14546 ts_tablet_manager.cc:1428] T 22d64cc3a2fe4f2fb091c16365dd0407 P acf697814b5e4390ae1e3c5faf1a4d62: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260808 13:04:22.581136 14456 raft_consensus.cc:482] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:04:22.581378 14456 raft_consensus.cc:504] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:22.583037 14456 leader_election.cc:288] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers acf697814b5e4390ae1e3c5faf1a4d62 (127.1.124.129:34901), 5daa346ccf3c406c9acda65c08d4fab6 (127.1.124.130:40305)
I20260808 13:04:22.592598 14555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:22.583357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:47203 (local address 127.1.124.129:34901)
0808 13:04:22.583526 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:22.583533 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:22.584295 (+ 762us) server_negotiation.cc:407] Connection header received
0808 13:04:22.584731 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:22.584740 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:22.584848 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:22.584947 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:22.585663 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.586807 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.589873 (+ 3066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.591157 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.591354 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:22.591437 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:22.592166 (+ 729us) server_negotiation.cc:299] Negotiation successful
0808 13:04:22.592321 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 13:04:22.592741 14579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:22.583291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:40305 (local address 127.1.124.131:56521)
0808 13:04:22.584693 (+ 1402us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:22.584713 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:22.584802 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:22.585145 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:22.585149 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:22.585203 (+ 54us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:22.585686 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:22.585697 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.587158 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.587163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:22.589698 (+ 2535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:22.589715 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.591547 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.591555 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:22.591863 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:22.591910 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 13:04:22.592018 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 13:04:22.592199 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1278,"mutex_wait_us":307,"spinlock_wait_cycles":6016,"thread_start_us":1020,"threads_started":1}
I20260808 13:04:22.592768 14578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:22.583291 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:34901 (local address 127.1.124.131:47203)
0808 13:04:22.584185 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:22.584205 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:22.584509 (+ 304us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:22.584974 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:22.584977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:22.584996 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:22.585450 (+ 454us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:22.585470 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.586975 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.586979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:22.589698 (+ 2719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:22.589715 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.591495 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.591508 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:22.591849 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:22.591905 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 13:04:22.592086 (+ 181us) client_negotiation.cc:240] Negotiation successful
0808 13:04:22.592199 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":713,"mutex_wait_us":200,"thread_start_us":404,"threads_started":1}
I20260808 13:04:22.592657 14580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:22.583542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:56521 (local address 127.1.124.130:40305)
0808 13:04:22.584486 (+ 944us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:22.584492 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:22.584798 (+ 306us) server_negotiation.cc:407] Connection header received
0808 13:04:22.584898 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:22.584904 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:22.584978 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:22.585074 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:22.585873 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.587012 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.589882 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.591169 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.591384 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:22.591450 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:22.592166 (+ 716us) server_negotiation.cc:299] Negotiation successful
0808 13:04:22.592321 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":787,"thread_start_us":614,"threads_started":1}
I20260808 13:04:22.593578 13767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22d64cc3a2fe4f2fb091c16365dd0407" candidate_uuid: "ebee817e0f46434d89c19aea5e53fa04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" is_pre_election: true
I20260808 13:04:22.593871 14012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22d64cc3a2fe4f2fb091c16365dd0407" candidate_uuid: "ebee817e0f46434d89c19aea5e53fa04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5daa346ccf3c406c9acda65c08d4fab6" is_pre_election: true
I20260808 13:04:22.594095 13767 raft_consensus.cc:2454] T 22d64cc3a2fe4f2fb091c16365dd0407 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebee817e0f46434d89c19aea5e53fa04 in term 0.
I20260808 13:04:22.594125 14012 raft_consensus.cc:2454] T 22d64cc3a2fe4f2fb091c16365dd0407 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ebee817e0f46434d89c19aea5e53fa04 in term 0.
I20260808 13:04:22.594753 14141 leader_election.cc:302] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5daa346ccf3c406c9acda65c08d4fab6, acf697814b5e4390ae1e3c5faf1a4d62, ebee817e0f46434d89c19aea5e53fa04; no voters:
I20260808 13:04:22.595208 14456 raft_consensus.cc:2781] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:04:22.595359 14456 raft_consensus.cc:482] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:04:22.595471 14456 raft_consensus.cc:3037] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:22.597790 14456 raft_consensus.cc:504] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:22.598426 14456 leader_election.cc:288] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04 [CANDIDATE]: Term 1 election: Requested vote from peers acf697814b5e4390ae1e3c5faf1a4d62 (127.1.124.129:34901), 5daa346ccf3c406c9acda65c08d4fab6 (127.1.124.130:40305)
I20260808 13:04:22.598907 13767 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22d64cc3a2fe4f2fb091c16365dd0407" candidate_uuid: "ebee817e0f46434d89c19aea5e53fa04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acf697814b5e4390ae1e3c5faf1a4d62"
I20260808 13:04:22.598997 14012 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "22d64cc3a2fe4f2fb091c16365dd0407" candidate_uuid: "ebee817e0f46434d89c19aea5e53fa04" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5daa346ccf3c406c9acda65c08d4fab6"
I20260808 13:04:22.599198 13767 raft_consensus.cc:3037] T 22d64cc3a2fe4f2fb091c16365dd0407 P acf697814b5e4390ae1e3c5faf1a4d62 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:22.599254 14012 raft_consensus.cc:3037] T 22d64cc3a2fe4f2fb091c16365dd0407 P 5daa346ccf3c406c9acda65c08d4fab6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:22.601539 14012 raft_consensus.cc:2454] T 22d64cc3a2fe4f2fb091c16365dd0407 P 5daa346ccf3c406c9acda65c08d4fab6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebee817e0f46434d89c19aea5e53fa04 in term 1.
I20260808 13:04:22.601711 13767 raft_consensus.cc:2454] T 22d64cc3a2fe4f2fb091c16365dd0407 P acf697814b5e4390ae1e3c5faf1a4d62 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ebee817e0f46434d89c19aea5e53fa04 in term 1.
I20260808 13:04:22.602252 14141 leader_election.cc:302] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5daa346ccf3c406c9acda65c08d4fab6, ebee817e0f46434d89c19aea5e53fa04; no voters:
I20260808 13:04:22.602706 14456 raft_consensus.cc:2781] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:04:22.603382 14456 raft_consensus.cc:686] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04 [term 1 LEADER]: Becoming Leader. State: Replica: ebee817e0f46434d89c19aea5e53fa04, State: Running, Role: LEADER
I20260808 13:04:22.603825 14456 consensus_queue.cc:237] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } }
I20260808 13:04:22.607833 13511 catalog_manager.cc:5495] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04 reported cstate change: term changed from 0 to 1, leader changed from <none> to ebee817e0f46434d89c19aea5e53fa04 (127.1.124.131). New cstate: current_term: 1 leader_uuid: "ebee817e0f46434d89c19aea5e53fa04" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ebee817e0f46434d89c19aea5e53fa04" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 39001 } health_report { overall_health: HEALTHY } } }
I20260808 13:04:22.641307 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:22.635087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50208 (local address 127.1.124.131:39001)
0808 13:04:22.635710 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:22.635715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:22.635733 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:04:22.635818 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:22.635821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:22.635893 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:22.636104 (+ 211us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:04:22.636585 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.637424 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.638798 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.639211 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.639353 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:22.639830 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:04:22.640570 (+ 740us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:04:22.640734 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:22.640859 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 13:04:22.640941 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489,"thread_start_us":334,"threads_started":1}
I20260808 13:04:22.658175 14555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:22.652446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:38615 (local address 127.1.124.129:34901)
0808 13:04:22.652729 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:22.652739 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:22.653126 (+ 387us) server_negotiation.cc:407] Connection header received
0808 13:04:22.653369 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:22.653373 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:22.653442 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:22.653544 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:04:22.654073 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.654870 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.656345 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.656751 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.656886 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:22.657350 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:04:22.657476 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:04:22.657578 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:22.657736 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 13:04:22.657822 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260808 13:04:22.658221 14592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:22.652174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:34901 (local address 127.1.124.131:38615)
0808 13:04:22.653007 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:22.653023 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:22.653213 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:22.653577 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:22.653581 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:22.653594 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:04:22.653905 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:22.653913 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.655026 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.655033 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:22.656202 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:22.656215 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.656944 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.656951 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:22.657097 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:22.657160 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:04:22.657588 (+ 428us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:04:22.657602 (+ 14us) client_negotiation.cc:769] Sending connection context
0808 13:04:22.657701 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 13:04:22.657838 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":453,"threads_started":1}
I20260808 13:04:22.667984 14012 raft_consensus.cc:1261] T 22d64cc3a2fe4f2fb091c16365dd0407 P 5daa346ccf3c406c9acda65c08d4fab6 [term 1 FOLLOWER]: Refusing update from remote peer ebee817e0f46434d89c19aea5e53fa04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:04:22.668139 13767 raft_consensus.cc:1261] T 22d64cc3a2fe4f2fb091c16365dd0407 P acf697814b5e4390ae1e3c5faf1a4d62 [term 1 FOLLOWER]: Refusing update from remote peer ebee817e0f46434d89c19aea5e53fa04: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:04:22.668708 14581 consensus_queue.cc:1035] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5daa346ccf3c406c9acda65c08d4fab6" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40305 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:04:22.669183 14456 consensus_queue.cc:1035] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acf697814b5e4390ae1e3c5faf1a4d62" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34901 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:04:22.716506 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:22.710698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:48321 (local address 127.1.124.131:39001)
0808 13:04:22.710903 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:22.710910 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:22.711719 (+ 809us) server_negotiation.cc:407] Connection header received
0808 13:04:22.711887 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:22.711893 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:22.711977 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:22.712109 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:04:22.712687 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.713537 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.714836 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.715268 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.715364 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:22.715719 (+ 355us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:04:22.715822 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:04:22.715893 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:22.716169 (+ 276us) server_negotiation.cc:299] Negotiation successful
0808 13:04:22.716261 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 13:04:22.716631 14599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:22.710620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:39001 (local address 127.1.124.129:48321)
0808 13:04:22.711578 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:22.711599 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:22.711749 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:22.712126 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:22.712131 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:22.712143 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 13:04:22.712462 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:22.712475 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.713660 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.713664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:22.714698 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:22.714709 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.715408 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.715412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:22.715535 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:22.715572 (+ 37us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 13:04:22.715955 (+ 383us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 13:04:22.715967 (+ 12us) client_negotiation.cc:769] Sending connection context
0808 13:04:22.716076 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 13:04:22.716223 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":786,"thread_start_us":621,"threads_started":1}
I20260808 13:04:22.753110 13511 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:04:22.753418 13511 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:04:22.756800 13511 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5bdcc074813a402d9f23a14899c0eb44: Sending DeleteTablet for 3 replicas of tablet 22d64cc3a2fe4f2fb091c16365dd0407
I20260808 13:04:22.757663 13734 tablet_service.cc:1514] Processing DeleteTablet for tablet 22d64cc3a2fe4f2fb091c16365dd0407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:04:22 UTC) from {username='oryx', principal='oryx/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:57314
I20260808 13:04:22.757756 13978 tablet_service.cc:1514] Processing DeleteTablet for tablet 22d64cc3a2fe4f2fb091c16365dd0407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:04:22 UTC) from {username='oryx', principal='oryx/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:51392
I20260808 13:04:22.757968 14569 tablet_replica.cc:330] T 22d64cc3a2fe4f2fb091c16365dd0407 P acf697814b5e4390ae1e3c5faf1a4d62: stopping tablet replica
I20260808 13:04:22.758055 14571 tablet_replica.cc:330] T 22d64cc3a2fe4f2fb091c16365dd0407 P 5daa346ccf3c406c9acda65c08d4fab6: stopping tablet replica
I20260808 13:04:22.758230 14569 raft_consensus.cc:2229] T 22d64cc3a2fe4f2fb091c16365dd0407 P acf697814b5e4390ae1e3c5faf1a4d62 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:04:22.758268 14571 raft_consensus.cc:2229] T 22d64cc3a2fe4f2fb091c16365dd0407 P 5daa346ccf3c406c9acda65c08d4fab6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:04:22.758442 14569 raft_consensus.cc:2258] T 22d64cc3a2fe4f2fb091c16365dd0407 P acf697814b5e4390ae1e3c5faf1a4d62 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:04:22.758459 14571 raft_consensus.cc:2258] T 22d64cc3a2fe4f2fb091c16365dd0407 P 5daa346ccf3c406c9acda65c08d4fab6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:04:22.758630 14221 tablet_service.cc:1514] Processing DeleteTablet for tablet 22d64cc3a2fe4f2fb091c16365dd0407 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:04:22 UTC) from {username='oryx', principal='oryx/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:50200
I20260808 13:04:22.758910 14572 tablet_replica.cc:330] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04: stopping tablet replica
I20260808 13:04:22.759119 14572 raft_consensus.cc:2229] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:04:22.759652 14572 raft_consensus.cc:2258] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:04:22.759763 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 13622
I20260808 13:04:22.759958 14571 ts_tablet_manager.cc:1905] T 22d64cc3a2fe4f2fb091c16365dd0407 P 5daa346ccf3c406c9acda65c08d4fab6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:22.761472 14572 ts_tablet_manager.cc:1905] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:22.766964 14571 ts_tablet_manager.cc:1918] T 22d64cc3a2fe4f2fb091c16365dd0407 P 5daa346ccf3c406c9acda65c08d4fab6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:04:22.767148 14571 log.cc:1196] T 22d64cc3a2fe4f2fb091c16365dd0407 P 5daa346ccf3c406c9acda65c08d4fab6: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/22d64cc3a2fe4f2fb091c16365dd0407
I20260808 13:04:22.767603 14571 ts_tablet_manager.cc:1939] T 22d64cc3a2fe4f2fb091c16365dd0407 P 5daa346ccf3c406c9acda65c08d4fab6: Deleting consensus metadata
I20260808 13:04:22.768798 13492 catalog_manager.cc:4841] TS 5daa346ccf3c406c9acda65c08d4fab6 (127.1.124.130:40305): tablet 22d64cc3a2fe4f2fb091c16365dd0407 (table test-table [id=af6b8fa956754956b6c654d849cf6127]) successfully deleted
I20260808 13:04:22.769189 14572 ts_tablet_manager.cc:1918] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:04:22.769420 14572 log.cc:1196] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/22d64cc3a2fe4f2fb091c16365dd0407
I20260808 13:04:22.769940 14569 ts_tablet_manager.cc:1905] T 22d64cc3a2fe4f2fb091c16365dd0407 P acf697814b5e4390ae1e3c5faf1a4d62: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:22.769989 14572 ts_tablet_manager.cc:1939] T 22d64cc3a2fe4f2fb091c16365dd0407 P ebee817e0f46434d89c19aea5e53fa04: Deleting consensus metadata
I20260808 13:04:22.771190 13493 catalog_manager.cc:4841] TS ebee817e0f46434d89c19aea5e53fa04 (127.1.124.131:39001): tablet 22d64cc3a2fe4f2fb091c16365dd0407 (table test-table [id=af6b8fa956754956b6c654d849cf6127]) successfully deleted
I20260808 13:04:22.776335 14569 ts_tablet_manager.cc:1918] T 22d64cc3a2fe4f2fb091c16365dd0407 P acf697814b5e4390ae1e3c5faf1a4d62: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 13:04:22.776502 14569 log.cc:1196] T 22d64cc3a2fe4f2fb091c16365dd0407 P acf697814b5e4390ae1e3c5faf1a4d62: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/22d64cc3a2fe4f2fb091c16365dd0407
I20260808 13:04:22.776963 14569 ts_tablet_manager.cc:1939] T 22d64cc3a2fe4f2fb091c16365dd0407 P acf697814b5e4390ae1e3c5faf1a4d62: Deleting consensus metadata
I20260808 13:04:22.778095 13489 catalog_manager.cc:4841] TS acf697814b5e4390ae1e3c5faf1a4d62 (127.1.124.129:34901): tablet 22d64cc3a2fe4f2fb091c16365dd0407 (table test-table [id=af6b8fa956754956b6c654d849cf6127]) successfully deleted
Aug 08 13:04:22 dist-test-slave-t3q3 krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.124.129@KRBTEST.COM
I20260808 13:04:22.789636 14555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:22.760641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57334 (local address 127.1.124.129:34901)
0808 13:04:22.760812 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:22.760818 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:22.772311 (+ 11493us) server_negotiation.cc:407] Connection header received
0808 13:04:22.772640 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:22.772648 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:22.772747 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:22.772866 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:22.774557 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.775743 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.777680 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:22.778364 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:22.778511 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:22.783361 (+ 4850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:22.783376 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:22.783393 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:22.783430 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:22.786652 (+ 3222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:22.787535 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:22.787542 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:22.787544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:22.787664 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:22.788162 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:22.788165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:22.788167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:22.788572 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:22.788796 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:22.789156 (+ 360us) server_negotiation.cc:299] Negotiation successful
0808 13:04:22.789345 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 13:04:22.790349 13840 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:22.949892 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 13622
I20260808 13:04:22.996865 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 13864
Aug 08 13:04:23 dist-test-slave-t3q3 krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.124.130@KRBTEST.COM
I20260808 13:04:23.014747 14580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:22.997754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51410 (local address 127.1.124.130:40305)
0808 13:04:22.997975 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:22.997983 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:22.998008 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:04:22.998211 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:22.998217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:22.998306 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:22.998419 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:22.999740 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:23.000742 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:23.002300 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:23.002733 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:23.002895 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:23.007483 (+ 4588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:23.007501 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:23.007524 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:23.007576 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:23.011493 (+ 3917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:23.012366 (+ 873us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:23.012374 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:23.012377 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:23.012564 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:23.012979 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:23.012986 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:23.012992 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:23.013469 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:23.013772 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:23.014058 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 13:04:23.014318 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 13:04:23.015580 14096 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:23.142769 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 13864
I20260808 13:04:23.182015 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 14108
Aug 08 13:04:23 dist-test-slave-t3q3 krb5kdc[13438](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.1.124.131@KRBTEST.COM
I20260808 13:04:23.200814 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:23.182917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50210 (local address 127.1.124.131:39001)
0808 13:04:23.183167 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:23.183171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:23.183190 (+ 19us) server_negotiation.cc:407] Connection header received
0808 13:04:23.183382 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:23.183386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:23.183451 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:23.183551 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:23.185452 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:23.186182 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:23.187966 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:23.188563 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:23.188669 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:23.194308 (+ 5639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:23.194320 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:23.194336 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:23.194374 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:23.197695 (+ 3321us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:23.198557 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:23.198565 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:23.198568 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:23.198701 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:23.199093 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:23.199096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:23.199098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:23.199505 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:23.199748 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:23.200181 (+ 433us) server_negotiation.cc:299] Negotiation successful
0808 13:04:23.200420 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260808 13:04:23.201639 14329 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:23.342507 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 14108
I20260808 13:04:23.373765 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 13457
I20260808 13:04:23.375041 13578 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:23.493043 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 13457
2026-08-08T13:04:23Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (8159 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:04:23 dist-test-slave-t3q3 krb5kdc[14616](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:04:23 dist-test-slave-t3q3 krb5kdc[14616](info): set up 2 sockets
Aug 08 13:04:23 dist-test-slave-t3q3 krb5kdc[14616](info): commencing operation
krb5kdc: starting...
W20260808 13:04:24.877489 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.333s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:04:25 dist-test-slave-t3q3 krb5kdc[14616](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194265, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:04:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:04:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:04:25.172856 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:44955
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:43337
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:44955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:25.517895 14639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:25.518314 14639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:25.518530 14639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:25.526597 14639 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:04:25.526710 14639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:25.526757 14639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:25.526816 14639 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:04:25.526865 14639 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:04:25.536952 14639 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:43337
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:44955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:44955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:25.538233 14639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:25.539558 14639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:25.551609 14646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:25.551651 14647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:25.551673 14650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:25.552758 14639 server_base.cc:1034] running on GCE node
I20260808 13:04:25.553608 14639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:25.555044 14639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:25.556284 14639 hybrid_clock.cc:648] HybridClock initialized: now 1786194265556248 us; error 55 us; skew 500 ppm
I20260808 13:04:25.556648 14639 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T13:04:25Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2127 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:04:25 dist-test-slave-t3q3 krb5kdc[14660](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:04:25 dist-test-slave-t3q3 krb5kdc[14660](info): set up 2 sockets
Aug 08 13:04:25 dist-test-slave-t3q3 krb5kdc[14660](info): commencing operation
krb5kdc: starting...
W20260808 13:04:26.983525 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.312s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:04:27 dist-test-slave-t3q3 krb5kdc[14660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:04:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:04:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:04:27.267761 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:38257
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:39901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:38257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:27.614116 14686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:27.614418 14686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:27.614600 14686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:27.622090 14686 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:04:27.622217 14686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:27.622328 14686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:27.622433 14686 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:04:27.622502 14686 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:04:27.632620 14686 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:39901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:38257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:38257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:27.634048 14686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:27.635557 14686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:27.648232 14693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:27.648343 14694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:27.648278 14698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:27.649119 14686 server_base.cc:1034] running on GCE node
I20260808 13:04:27.649917 14686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:27.651350 14686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:27.652592 14686 hybrid_clock.cc:648] HybridClock initialized: now 1786194267652532 us; error 48 us; skew 500 ppm
I20260808 13:04:27.652974 14686 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:27 dist-test-slave-t3q3 krb5kdc[14660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194267, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:27.659221 14686 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:04:27.660923 14686 webserver.cc:460] Webserver started at http://127.1.124.190:37727/ using document root <none> and password file <none>
I20260808 13:04:27.661578 14686 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:27.661679 14686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:27.661937 14686 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:27.664924 14686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "88184ae9b4e1442392e7e9b1eae6a31c"
format_stamp: "Formatted at 2026-08-08 13:04:27 on dist-test-slave-t3q3"
server_key: "4e5a00a23d75da54e565f33c0c954c2f"
server_key_iv: "0165079e3f30195853c6cdfbce6aa953"
server_key_version: "encryptionkey@0"
I20260808 13:04:27.665681 14686 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "88184ae9b4e1442392e7e9b1eae6a31c"
format_stamp: "Formatted at 2026-08-08 13:04:27 on dist-test-slave-t3q3"
server_key: "4e5a00a23d75da54e565f33c0c954c2f"
server_key_iv: "0165079e3f30195853c6cdfbce6aa953"
server_key_version: "encryptionkey@0"
I20260808 13:04:27.670885 14686 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:04:27.674360 14711 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:27.675720 14686 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:04:27.675942 14686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "88184ae9b4e1442392e7e9b1eae6a31c"
format_stamp: "Formatted at 2026-08-08 13:04:27 on dist-test-slave-t3q3"
server_key: "4e5a00a23d75da54e565f33c0c954c2f"
server_key_iv: "0165079e3f30195853c6cdfbce6aa953"
server_key_version: "encryptionkey@0"
I20260808 13:04:27.676246 14686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:27.692369 14686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:27.697275 14686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:27.697589 14686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:27.717693 14686 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:38257
I20260808 13:04:27.717712 14801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:38257 every 8 connection(s)
I20260808 13:04:27.719213 14686 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:04:27.723290 14802 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:27.727412 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 14686
I20260808 13:04:27.727689 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:04:27.728560 1522 external_mini_cluster.cc:1428] Setting key 64702a88175ff07ecf4fd91626bf6605
I20260808 13:04:27.735241 14802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c: Bootstrap starting.
I20260808 13:04:27.739169 14802 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:27.740525 14802 log.cc:824] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c: Log is configured to *not* fsync() on all Append() calls
I20260808 13:04:27.744313 14802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c: No bootstrap required, opened a new log
Aug 08 13:04:27 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:04:27.753746 14802 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88184ae9b4e1442392e7e9b1eae6a31c" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38257 } }
I20260808 13:04:27.754161 14802 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:27.754292 14802 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88184ae9b4e1442392e7e9b1eae6a31c, State: Initialized, Role: FOLLOWER
I20260808 13:04:27.754993 14806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:27.730670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32890 (local address 127.1.124.190:38257)
0808 13:04:27.731517 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:27.731530 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:27.731581 (+ 51us) server_negotiation.cc:407] Connection header received
0808 13:04:27.733634 (+ 2053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:27.733660 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:27.734071 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:27.734404 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:27.736480 (+ 2076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:27.737929 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:27.739839 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:27.740769 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:27.740932 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:27.746201 (+ 5269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:27.746210 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:27.746229 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:27.746274 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:27.749749 (+ 3475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:27.750650 (+ 901us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:27.750658 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:27.750665 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:27.750876 (+ 211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:27.751413 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:27.751418 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:27.751420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:27.752290 (+ 870us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:27.752577 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:27.753102 (+ 525us) server_negotiation.cc:299] Negotiation successful
0808 13:04:27.753619 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":387,"threads_started":1}
I20260808 13:04:27.754889 14802 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88184ae9b4e1442392e7e9b1eae6a31c" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38257 } }
I20260808 13:04:27.755174 14802 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:04:27.755360 14802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:04:27.755581 14802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:27.758338 14802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88184ae9b4e1442392e7e9b1eae6a31c" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38257 } }
I20260808 13:04:27.758903 14802 leader_election.cc:302] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88184ae9b4e1442392e7e9b1eae6a31c; no voters:
I20260808 13:04:27.759502 14802 leader_election.cc:288] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:04:27.759665 14808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:04:27.760464 14808 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [term 1 LEADER]: Becoming Leader. State: Replica: 88184ae9b4e1442392e7e9b1eae6a31c, State: Running, Role: LEADER
I20260808 13:04:27.761011 14808 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88184ae9b4e1442392e7e9b1eae6a31c" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38257 } }
I20260808 13:04:27.761684 14802 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:04:27.763732 14810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88184ae9b4e1442392e7e9b1eae6a31c. Latest consensus state: current_term: 1 leader_uuid: "88184ae9b4e1442392e7e9b1eae6a31c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88184ae9b4e1442392e7e9b1eae6a31c" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38257 } } }
I20260808 13:04:27.764395 14810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:27.763759 14809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88184ae9b4e1442392e7e9b1eae6a31c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88184ae9b4e1442392e7e9b1eae6a31c" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38257 } } }
I20260808 13:04:27.764670 14809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:27.767424 14814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:04:27.774052 14814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:04:27.784960 14814 catalog_manager.cc:1347] Generated new cluster ID: e55838544e6f4203920bbfe0989f0869
I20260808 13:04:27.785118 14814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:04:27.809222 14814 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:04:27.810652 14814 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:04:27.820096 14814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c: Generated new TSK 0
I20260808 13:04:27.821233 14814 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.124.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
I20260808 13:04:27.896020 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:38257
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:39901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:28.238103 14840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:28.238437 14840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:28.238617 14840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:28.245982 14840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:28.246150 14840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:28.246328 14840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:04:28.256582 14840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:39901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:38257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:28.257951 14840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:28.259389 14840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:28.273432 14847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:28.273386 14851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:28.273471 14846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:28.274727 14840 server_base.cc:1034] running on GCE node
I20260808 13:04:28.275349 14840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:28.276351 14840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:28.277581 14840 hybrid_clock.cc:648] HybridClock initialized: now 1786194268277532 us; error 60 us; skew 500 ppm
I20260808 13:04:28.277907 14840 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:28 dist-test-slave-t3q3 krb5kdc[14660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194268, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:28.281375 14840 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:04:28.282881 14840 webserver.cc:460] Webserver started at http://127.1.124.129:43691/ using document root <none> and password file <none>
I20260808 13:04:28.283634 14840 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:28.283740 14840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:28.284005 14840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:28.286989 14840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "1773e6be24c74ec0af279645ed1b7548"
format_stamp: "Formatted at 2026-08-08 13:04:28 on dist-test-slave-t3q3"
server_key: "83b0da64446057eb4e8356d15035ea7a"
server_key_iv: "de26497c38c0cbe1cef833d8cfc34fab"
server_key_version: "encryptionkey@0"
I20260808 13:04:28.287752 14840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "1773e6be24c74ec0af279645ed1b7548"
format_stamp: "Formatted at 2026-08-08 13:04:28 on dist-test-slave-t3q3"
server_key: "83b0da64446057eb4e8356d15035ea7a"
server_key_iv: "de26497c38c0cbe1cef833d8cfc34fab"
server_key_version: "encryptionkey@0"
I20260808 13:04:28.293529 14840 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260808 13:04:28.297219 14863 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:28.298643 14840 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 13:04:28.298782 14840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "1773e6be24c74ec0af279645ed1b7548"
format_stamp: "Formatted at 2026-08-08 13:04:28 on dist-test-slave-t3q3"
server_key: "83b0da64446057eb4e8356d15035ea7a"
server_key_iv: "de26497c38c0cbe1cef833d8cfc34fab"
server_key_version: "encryptionkey@0"
I20260808 13:04:28.298996 14840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:28.325156 14840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:28.329768 14840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:28.330027 14840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:28.330973 14840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:28.332365 14840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:28.332441 14840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:28.332537 14840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:28.332583 14840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:28.368165 14840 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:33267
I20260808 13:04:28.368305 15062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:33267 every 8 connection(s)
I20260808 13:04:28.369639 14840 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:04:28.375551 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 14840
I20260808 13:04:28.375792 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:04:28.376690 1522 external_mini_cluster.cc:1428] Setting key a99af04e6e4a7dc164a97cfb7a1fc050
Aug 08 13:04:28 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194268, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:04:28.393013 15067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:28.373194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:49407 (local address 127.1.124.190:38257)
0808 13:04:28.373714 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:28.373720 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:28.374719 (+ 999us) server_negotiation.cc:407] Connection header received
0808 13:04:28.376983 (+ 2264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:28.376989 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:28.377082 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:28.377215 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:28.379239 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:28.380446 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:28.382127 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:28.382831 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:28.382979 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:28.386381 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:28.386391 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:28.386398 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:28.386442 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:28.389603 (+ 3161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:28.390587 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:28.390591 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:28.390594 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:28.390735 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:28.391244 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:28.391248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:28.391250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:28.391502 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:28.391613 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:28.392571 (+ 958us) server_negotiation.cc:299] Negotiation successful
0808 13:04:28.392749 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":263,"threads_started":1}
I20260808 13:04:28.394593 15068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:28.373801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.129:49407)
0808 13:04:28.374547 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:28.374594 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:28.376633 (+ 2039us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:28.377478 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:28.377489 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:28.378250 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:28.379056 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:28.379077 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:28.380602 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:28.380606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:28.381971 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:28.381984 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:28.383041 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:28.383045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:28.383234 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:28.383937 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:28.383960 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:28.386158 (+ 2198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:28.389766 (+ 3608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:28.389777 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:28.389799 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:28.390455 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:28.390858 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:28.390864 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:28.390869 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:28.391130 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:28.391645 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:28.391660 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:28.392155 (+ 495us) client_negotiation.cc:769] Sending connection context
0808 13:04:28.392465 (+ 310us) client_negotiation.cc:240] Negotiation successful
0808 13:04:28.393063 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":285,"threads_started":1}
I20260808 13:04:28.396363 15063 heartbeater.cc:347] Connected to a master server at 127.1.124.190:38257
I20260808 13:04:28.396725 15063 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:28.397541 15063 heartbeater.cc:511] Master 127.1.124.190:38257 requested a full tablet report, sending...
I20260808 13:04:28.399845 14738 ts_manager.cc:194] Registered new tserver with Master: 1773e6be24c74ec0af279645ed1b7548 (127.1.124.129:33267)
I20260808 13:04:28.401536 14738 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:49407
WARNING: no policy specified for kudu/127.1.124.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
I20260808 13:04:28.439738 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:38257
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:39901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:28.781452 15073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:28.781741 15073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:28.781903 15073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:28.789426 15073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:28.789565 15073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:28.789764 15073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:04:28.799636 15073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:39901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:38257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:28.800969 15073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:28.802276 15073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:28.815944 15079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:28.815896 15081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:28.815851 15086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:28.816551 15073 server_base.cc:1034] running on GCE node
I20260808 13:04:28.817250 15073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:28.818224 15073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:28.819481 15073 hybrid_clock.cc:648] HybridClock initialized: now 1786194268819437 us; error 49 us; skew 500 ppm
I20260808 13:04:28.819846 15073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:28 dist-test-slave-t3q3 krb5kdc[14660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194268, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:28.823318 15073 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:04:28.824927 15073 webserver.cc:460] Webserver started at http://127.1.124.130:37641/ using document root <none> and password file <none>
I20260808 13:04:28.825664 15073 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:28.825819 15073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:28.826189 15073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:28.829454 15073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "9fd162f119b745398bc569cfd338e42e"
format_stamp: "Formatted at 2026-08-08 13:04:28 on dist-test-slave-t3q3"
server_key: "5410aa1e3b524301235394857b1662ef"
server_key_iv: "0ad30f61a938cea9aabdc82159911eb3"
server_key_version: "encryptionkey@0"
I20260808 13:04:28.830261 15073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "9fd162f119b745398bc569cfd338e42e"
format_stamp: "Formatted at 2026-08-08 13:04:28 on dist-test-slave-t3q3"
server_key: "5410aa1e3b524301235394857b1662ef"
server_key_iv: "0ad30f61a938cea9aabdc82159911eb3"
server_key_version: "encryptionkey@0"
I20260808 13:04:28.835860 15073 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 13:04:28.839974 15102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:28.841516 15073 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:04:28.841730 15073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "9fd162f119b745398bc569cfd338e42e"
format_stamp: "Formatted at 2026-08-08 13:04:28 on dist-test-slave-t3q3"
server_key: "5410aa1e3b524301235394857b1662ef"
server_key_iv: "0ad30f61a938cea9aabdc82159911eb3"
server_key_version: "encryptionkey@0"
I20260808 13:04:28.841961 15073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:28.858038 15073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:28.862355 15073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:28.862679 15073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:28.863713 15073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:28.865127 15073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:28.865235 15073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:28.865392 15073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:28.865470 15073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:28.901721 15073 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:41185
I20260808 13:04:28.901759 15274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:41185 every 8 connection(s)
I20260808 13:04:28.903522 15073 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:04:28.908370 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 15073
I20260808 13:04:28.908593 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:04:28.909073 1522 external_mini_cluster.cc:1428] Setting key 7e3a80341178692b0979beaf513c48c5
Aug 08 13:04:28 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194268, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:04:28.931493 15279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:28.910942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:51555 (local address 127.1.124.190:38257)
0808 13:04:28.915050 (+ 4108us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:28.915059 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:28.915082 (+ 23us) server_negotiation.cc:407] Connection header received
0808 13:04:28.915155 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:28.915162 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:28.915253 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:28.915364 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:28.917097 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:28.918244 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:28.919784 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:28.920552 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:28.920703 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:28.924171 (+ 3468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:28.924178 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:28.924184 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:28.924229 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:28.927904 (+ 3675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:28.928731 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:28.928740 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:28.928746 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:28.928927 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:28.929379 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:28.929387 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:28.929393 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:28.929797 (+ 404us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:28.929956 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:28.930719 (+ 763us) server_negotiation.cc:299] Negotiation successful
0808 13:04:28.930990 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3963,"thread_start_us":3864,"threads_started":1}
I20260808 13:04:28.932126 15277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:28.909828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.130:51555)
0808 13:04:28.910745 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:28.910796 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:28.913610 (+ 2814us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:28.915682 (+ 2072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:28.915691 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:28.916330 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:28.916911 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:28.916930 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:28.918401 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:28.918404 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:28.919642 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:28.919654 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:28.920764 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:28.920768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:28.920967 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:28.921684 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:28.921706 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:28.923887 (+ 2181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:28.928133 (+ 4246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:28.928142 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:28.928157 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:28.928603 (+ 446us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:28.929047 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:28.929050 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:28.929052 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:28.929250 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:28.929923 (+ 673us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:28.929928 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:28.930288 (+ 360us) client_negotiation.cc:769] Sending connection context
0808 13:04:28.930529 (+ 241us) client_negotiation.cc:240] Negotiation successful
0808 13:04:28.930938 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":914,"threads_started":1}
I20260808 13:04:28.933706 15275 heartbeater.cc:347] Connected to a master server at 127.1.124.190:38257
I20260808 13:04:28.934036 15275 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:28.934759 15275 heartbeater.cc:511] Master 127.1.124.190:38257 requested a full tablet report, sending...
I20260808 13:04:28.936314 14738 ts_manager.cc:194] Registered new tserver with Master: 9fd162f119b745398bc569cfd338e42e (127.1.124.130:41185)
I20260808 13:04:28.937177 14738 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:51555
WARNING: no policy specified for kudu/127.1.124.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
I20260808 13:04:28.977784 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:38257
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:39901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:29.324900 15284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:29.325274 15284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:29.325460 15284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:29.332782 15284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:29.332950 15284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:29.333168 15284 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:04:29.343084 15284 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:39901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:38257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:29.344549 15284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:29.345963 15284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:29.360146 15293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:29.360188 15297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:29.360196 15294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:29.360791 15284 server_base.cc:1034] running on GCE node
I20260808 13:04:29.361505 15284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:29.362500 15284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:29.363749 15284 hybrid_clock.cc:648] HybridClock initialized: now 1786194269363693 us; error 64 us; skew 500 ppm
I20260808 13:04:29.364176 15284 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:29 dist-test-slave-t3q3 krb5kdc[14660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194269, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:29.367745 15284 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:04:29.370343 15284 webserver.cc:460] Webserver started at http://127.1.124.131:39265/ using document root <none> and password file <none>
I20260808 13:04:29.371294 15284 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:29.371438 15284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:29.371809 15284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:29.375773 15284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "55c0037008884da38873ec9e6f80cf41"
format_stamp: "Formatted at 2026-08-08 13:04:29 on dist-test-slave-t3q3"
server_key: "e99375380c2157cbfb754806f731255d"
server_key_iv: "cdc0049191208d9d40b233cf1b1e1d2a"
server_key_version: "encryptionkey@0"
I20260808 13:04:29.376950 15284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "55c0037008884da38873ec9e6f80cf41"
format_stamp: "Formatted at 2026-08-08 13:04:29 on dist-test-slave-t3q3"
server_key: "e99375380c2157cbfb754806f731255d"
server_key_iv: "cdc0049191208d9d40b233cf1b1e1d2a"
server_key_version: "encryptionkey@0"
I20260808 13:04:29.382375 15284 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:04:29.385993 15306 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:29.387398 15284 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:04:29.387557 15284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "55c0037008884da38873ec9e6f80cf41"
format_stamp: "Formatted at 2026-08-08 13:04:29 on dist-test-slave-t3q3"
server_key: "e99375380c2157cbfb754806f731255d"
server_key_iv: "cdc0049191208d9d40b233cf1b1e1d2a"
server_key_version: "encryptionkey@0"
I20260808 13:04:29.387727 15284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:29.405661 15063 heartbeater.cc:503] Master 127.1.124.190:38257 was elected leader, sending a full tablet report...
I20260808 13:04:29.420853 15284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:29.424962 15284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:29.425280 15284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:29.426283 15284 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:29.427644 15284 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:29.427719 15284 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:29.427901 15284 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:29.427963 15284 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:29.463129 15284 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:36891
I20260808 13:04:29.463231 15494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:36891 every 8 connection(s)
I20260808 13:04:29.464540 15284 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:04:29.466246 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 15284
I20260808 13:04:29.466455 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:04:29.467057 1522 external_mini_cluster.cc:1428] Setting key c3b95f12260b7de1d15f622cdd1b0f77
Aug 08 13:04:29 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194269, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:04:29.485652 15497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:29.467336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:50817 (local address 127.1.124.190:38257)
0808 13:04:29.467923 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:29.467929 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:29.469203 (+ 1274us) server_negotiation.cc:407] Connection header received
0808 13:04:29.471464 (+ 2261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:29.471469 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:29.471572 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:29.471662 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:29.473214 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:29.474018 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:29.475582 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:29.476026 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:29.476198 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:29.479663 (+ 3465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:29.479668 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:29.479672 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:29.479702 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:29.482896 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:29.483620 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:29.483624 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:29.483626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:29.483748 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:29.484184 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:29.484187 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:29.484189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:29.484450 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:29.484576 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:29.485206 (+ 630us) server_negotiation.cc:299] Negotiation successful
0808 13:04:29.485381 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":272,"threads_started":1}
I20260808 13:04:29.486593 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:29.467991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.131:50817)
0808 13:04:29.469005 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:29.469096 (+ 91us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:29.471210 (+ 2114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:29.471877 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:29.471885 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:29.472486 (+ 601us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:29.473046 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:29.473066 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:29.474177 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:29.474181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:29.475427 (+ 1246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:29.475439 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:29.476297 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:29.476302 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:29.476493 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:29.477157 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:29.477178 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:29.479435 (+ 2257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:29.483061 (+ 3626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:29.483068 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:29.483079 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:29.483511 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:29.483844 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:29.483847 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:29.483849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:29.484023 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:29.484555 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:29.484560 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:29.484857 (+ 297us) client_negotiation.cc:769] Sending connection context
0808 13:04:29.485075 (+ 218us) client_negotiation.cc:240] Negotiation successful
0808 13:04:29.485466 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"thread_start_us":362,"threads_started":1}
I20260808 13:04:29.487959 15495 heartbeater.cc:347] Connected to a master server at 127.1.124.190:38257
I20260808 13:04:29.488312 15495 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:29.489037 15495 heartbeater.cc:511] Master 127.1.124.190:38257 requested a full tablet report, sending...
I20260808 13:04:29.490449 14738 ts_manager.cc:194] Registered new tserver with Master: 55c0037008884da38873ec9e6f80cf41 (127.1.124.131:36891)
I20260808 13:04:29.491384 14738 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:50817
I20260808 13:04:29.495263 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.1.124.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.124.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.124.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.129.keytab.
Entry for principal oryx/127.1.124.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.129.keytab.
WARNING: no policy specified for oryx/127.1.124.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.124.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.124.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.130.keytab.
Entry for principal oryx/127.1.124.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.130.keytab.
WARNING: no policy specified for oryx/127.1.124.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.1.124.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.1.124.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.131.keytab.
Entry for principal oryx/127.1.124.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.131.keytab.
I20260808 13:04:29.667317 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 14840
Aug 08 13:04:29 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.129@KRBTEST.COM
I20260808 13:04:29.684520 15511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:29.668600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48920 (local address 127.1.124.129:33267)
0808 13:04:29.669388 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:29.669397 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:29.669424 (+ 27us) server_negotiation.cc:407] Connection header received
0808 13:04:29.669493 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:29.669500 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:29.669795 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:29.670016 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:29.671427 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:29.672578 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:29.673871 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:29.674462 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:29.674576 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:29.678549 (+ 3973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:29.678561 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:29.678576 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:29.678618 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:29.681884 (+ 3266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:29.682536 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:29.682544 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:29.682546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:29.682686 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:29.683050 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:29.683054 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:29.683056 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:29.683402 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:29.683643 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:29.683859 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 13:04:29.684188 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":457,"threads_started":1}
I20260808 13:04:29.685828 15055 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:29.769956 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 14840
I20260808 13:04:29.793754 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 15073
Aug 08 13:04:29 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.130@KRBTEST.COM
I20260808 13:04:29.809695 15516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:29.794711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42642 (local address 127.1.124.130:41185)
0808 13:04:29.795315 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:29.795322 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:29.795346 (+ 24us) server_negotiation.cc:407] Connection header received
0808 13:04:29.795404 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:29.795410 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:29.795626 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:29.795823 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:29.797216 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:29.798224 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:29.799524 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:29.800245 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:29.800381 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:29.804519 (+ 4138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:29.804530 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:29.804545 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:29.804576 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:29.807262 (+ 2686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:29.807876 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:29.807882 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:29.807884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:29.808003 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:29.808394 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:29.808397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:29.808400 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:29.808717 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:29.808935 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:29.809222 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 13:04:29.809413 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":350,"threads_started":1}
I20260808 13:04:29.811048 15270 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:29.893990 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 15073
I20260808 13:04:29.917090 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 15284
Aug 08 13:04:29 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.131@KRBTEST.COM
I20260808 13:04:29.933759 15518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:29.918205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59910 (local address 127.1.124.131:36891)
0808 13:04:29.918722 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:29.918729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:29.918760 (+ 31us) server_negotiation.cc:407] Connection header received
0808 13:04:29.918821 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:29.918827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:29.919049 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:29.919175 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:29.920549 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:29.921533 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:29.922844 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:29.923398 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:29.923497 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:29.927953 (+ 4456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:29.927964 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:29.927980 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:29.928012 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:29.930836 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:29.931599 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:29.931606 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:29.931608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:29.931739 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:29.932237 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:29.932241 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:29.932242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:29.932597 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:29.932832 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:29.933260 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 13:04:29.933461 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":227,"threads_started":1}
I20260808 13:04:29.935160 15486 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:30.019407 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 15284
I20260808 13:04:30.042533 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 14686
I20260808 13:04:30.043833 14794 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:30.147246 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 14686
I20260808 13:04:30.172681 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:38257
--webserver_interface=127.1.124.190
--webserver_port=37727
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:39901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:38257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:30.517513 15522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:30.517807 15522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:30.517993 15522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:30.525295 15522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:04:30.525418 15522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:30.525521 15522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:30.525619 15522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:04:30.525707 15522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:04:30.535657 15522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:39901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:38257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:38257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=37727
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:30.537082 15522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:30.538461 15522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:30.550825 15534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:30.550879 15537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:30.550901 15533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:30.551466 15522 server_base.cc:1034] running on GCE node
I20260808 13:04:30.552420 15522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:30.554018 15522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:30.555294 15522 hybrid_clock.cc:648] HybridClock initialized: now 1786194270555247 us; error 41 us; skew 500 ppm
I20260808 13:04:30.555644 15522 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:30 dist-test-slave-t3q3 krb5kdc[14660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194270, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:30.559041 15522 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:04:30.560652 15522 webserver.cc:460] Webserver started at http://127.1.124.190:37727/ using document root <none> and password file <none>
I20260808 13:04:30.561345 15522 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:30.561476 15522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:30.567318 15522 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 13:04:30.570456 15553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:30.571908 15522 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:04:30.572114 15522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "88184ae9b4e1442392e7e9b1eae6a31c"
format_stamp: "Formatted at 2026-08-08 13:04:27 on dist-test-slave-t3q3"
server_key: "4e5a00a23d75da54e565f33c0c954c2f"
server_key_iv: "0165079e3f30195853c6cdfbce6aa953"
server_key_version: "encryptionkey@0"
I20260808 13:04:30.573132 15522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:30.592648 15522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:30.597638 15522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:30.597968 15522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:30.618002 15522 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:38257
I20260808 13:04:30.618023 15630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:38257 every 8 connection(s)
I20260808 13:04:30.619714 15522 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:04:30.626844 15631 sys_catalog.cc:263] Verifying existing consensus state
I20260808 13:04:30.630211 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 15522
I20260808 13:04:30.630452 15631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c: Bootstrap starting.
I20260808 13:04:30.631254 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:33267
--local_ip_for_outbound_sockets=127.1.124.129
--tserver_master_addrs=127.1.124.190:38257
--webserver_port=43691
--webserver_interface=127.1.124.129
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:39901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.124.129
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.129.keytab with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:04:30.653824 15631 log.cc:824] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c: Log is configured to *not* fsync() on all Append() calls
I20260808 13:04:30.662966 15631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 13:04:30.663587 15631 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c: Bootstrap complete.
I20260808 13:04:30.672976 15631 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88184ae9b4e1442392e7e9b1eae6a31c" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38257 } }
I20260808 13:04:30.673888 15631 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88184ae9b4e1442392e7e9b1eae6a31c, State: Initialized, Role: FOLLOWER
I20260808 13:04:30.674544 15631 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88184ae9b4e1442392e7e9b1eae6a31c" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38257 } }
I20260808 13:04:30.674790 15631 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:04:30.674924 15631 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:04:30.675123 15631 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:04:30.679235 15631 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88184ae9b4e1442392e7e9b1eae6a31c" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38257 } }
I20260808 13:04:30.679751 15631 leader_election.cc:302] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88184ae9b4e1442392e7e9b1eae6a31c; no voters:
I20260808 13:04:30.680452 15631 leader_election.cc:288] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 13:04:30.680666 15639 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [term 2 FOLLOWER]: Leader election won for term 2
I20260808 13:04:30.681869 15639 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [term 2 LEADER]: Becoming Leader. State: Replica: 88184ae9b4e1442392e7e9b1eae6a31c, State: Running, Role: LEADER
I20260808 13:04:30.682577 15631 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:04:30.682549 15639 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88184ae9b4e1442392e7e9b1eae6a31c" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38257 } }
I20260808 13:04:30.687706 15643 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "88184ae9b4e1442392e7e9b1eae6a31c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88184ae9b4e1442392e7e9b1eae6a31c" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38257 } } }
I20260808 13:04:30.688158 15643 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:30.688717 15646 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88184ae9b4e1442392e7e9b1eae6a31c. Latest consensus state: current_term: 2 leader_uuid: "88184ae9b4e1442392e7e9b1eae6a31c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88184ae9b4e1442392e7e9b1eae6a31c" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38257 } } }
I20260808 13:04:30.689023 15646 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:30.695694 15652 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:04:30.704327 15652 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:04:30.705945 15652 catalog_manager.cc:1259] Loaded cluster ID: e55838544e6f4203920bbfe0989f0869
I20260808 13:04:30.706267 15652 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:04:30.710476 15652 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:04:30.711964 15652 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 88184ae9b4e1442392e7e9b1eae6a31c: Loaded TSK: 0
I20260808 13:04:30.713544 15652 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 13:04:30.991803 15634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:30.992153 15634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:30.992328 15634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:30.999692 15634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:30.999838 15634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:31.000031 15634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:04:31.010295 15634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:39901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.129.keytab
--principal=oryx/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:33267
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=43691
--tserver_master_addrs=127.1.124.190:38257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:31.011642 15634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:31.013049 15634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:31.027581 15678 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:31.027685 15679 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:31.028525 15682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:31.029438 15634 server_base.cc:1034] running on GCE node
I20260808 13:04:31.030135 15634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:31.031198 15634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:31.032446 15634 hybrid_clock.cc:648] HybridClock initialized: now 1786194271032388 us; error 49 us; skew 500 ppm
I20260808 13:04:31.032800 15634 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:31 dist-test-slave-t3q3 krb5kdc[14660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194271, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:31.036437 15634 init.cc:377] Logged in from keytab as oryx/127.1.124.129@KRBTEST.COM (short username oryx)
I20260808 13:04:31.038255 15634 webserver.cc:460] Webserver started at http://127.1.124.129:43691/ using document root <none> and password file <none>
I20260808 13:04:31.039001 15634 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:31.039178 15634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:31.045060 15634 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 13:04:31.047935 15690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:31.049497 15634 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:04:31.049686 15634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "1773e6be24c74ec0af279645ed1b7548"
format_stamp: "Formatted at 2026-08-08 13:04:28 on dist-test-slave-t3q3"
server_key: "83b0da64446057eb4e8356d15035ea7a"
server_key_iv: "de26497c38c0cbe1cef833d8cfc34fab"
server_key_version: "encryptionkey@0"
I20260808 13:04:31.050686 15634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:31.070721 15634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:31.074937 15634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:31.075237 15634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:31.076288 15634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:31.077769 15634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:31.077885 15634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:31.078015 15634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:31.078125 15634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:31.112767 15634 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:33267
I20260808 13:04:31.112859 15889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:33267 every 8 connection(s)
I20260808 13:04:31.114274 15634 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:04:31.122354 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 15634
I20260808 13:04:31.123014 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:41185
--local_ip_for_outbound_sockets=127.1.124.130
--tserver_master_addrs=127.1.124.190:38257
--webserver_port=37641
--webserver_interface=127.1.124.130
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:39901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.124.130
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.130.keytab with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 13:04:31 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:31 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 13:04:31.468416 15894 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:31.468674 15894 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:31.468824 15894 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:31.475971 15894 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:31.476122 15894 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:31.476312 15894 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:04:31.486243 15894 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:39901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.130.keytab
--principal=oryx/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:41185
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=37641
--tserver_master_addrs=127.1.124.190:38257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:31.487572 15894 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:31.488952 15894 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:31.507380 15911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:31.507686 15914 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:31.508685 15910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:31.509814 15894 server_base.cc:1034] running on GCE node
I20260808 13:04:31.511070 15894 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:31.512792 15894 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:31.514115 15894 hybrid_clock.cc:648] HybridClock initialized: now 1786194271514026 us; error 47 us; skew 500 ppm
I20260808 13:04:31.514669 15894 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:31 dist-test-slave-t3q3 krb5kdc[14660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194271, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:31.520560 15894 init.cc:377] Logged in from keytab as oryx/127.1.124.130@KRBTEST.COM (short username oryx)
I20260808 13:04:31.523043 15894 webserver.cc:460] Webserver started at http://127.1.124.130:37641/ using document root <none> and password file <none>
I20260808 13:04:31.524394 15894 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:31.524622 15894 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:31.534936 15894 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.005s sys 0.000s
I20260808 13:04:31.539535 15928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:31.541812 15894 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 13:04:31.542153 15894 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "9fd162f119b745398bc569cfd338e42e"
format_stamp: "Formatted at 2026-08-08 13:04:28 on dist-test-slave-t3q3"
server_key: "5410aa1e3b524301235394857b1662ef"
server_key_iv: "0ad30f61a938cea9aabdc82159911eb3"
server_key_version: "encryptionkey@0"
I20260808 13:04:31.544181 15894 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:31.571833 15894 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:31.576591 15894 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:31.576862 15894 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:31.577983 15894 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:31.579439 15894 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:31.579566 15894 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:31.579698 15894 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:31.579785 15894 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:31.625252 15894 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:41185
I20260808 13:04:31.625298 16111 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:41185 every 8 connection(s)
I20260808 13:04:31.626832 15894 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:04:31.633353 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 15894
I20260808 13:04:31.634399 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:36891
--local_ip_for_outbound_sockets=127.1.124.131
--tserver_master_addrs=127.1.124.190:38257
--webserver_port=39265
--webserver_interface=127.1.124.131
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:39901
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.1.124.131
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.131.keytab with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 13:04:31 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:31 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 13:04:32.337872 16119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:32.338301 16119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:32.338546 16119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:32.349303 16119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:32.349510 16119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:32.349773 16119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:04:32.365137 16119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:39901
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/krb5kdc/oryx_127.1.124.131.keytab
--principal=oryx/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:36891
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=39265
--tserver_master_addrs=127.1.124.190:38257
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:32.367094 16119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:32.368942 16119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:32.390900 16127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:32.391337 16119 server_base.cc:1034] running on GCE node
W20260808 13:04:32.392845 16126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:32.393033 16129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:32.394172 16119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:32.395370 16119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:32.396653 16119 hybrid_clock.cc:648] HybridClock initialized: now 1786194272396589 us; error 56 us; skew 500 ppm
I20260808 13:04:32.397037 16119 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:32 dist-test-slave-t3q3 krb5kdc[14660](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194272, etypes {rep=17 tkt=17 ses=17}, oryx/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:32.401026 16119 init.cc:377] Logged in from keytab as oryx/127.1.124.131@KRBTEST.COM (short username oryx)
I20260808 13:04:32.403554 16119 webserver.cc:460] Webserver started at http://127.1.124.131:39265/ using document root <none> and password file <none>
I20260808 13:04:32.404927 16119 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:32.405145 16119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:32.415807 16119 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.006s sys 0.001s
I20260808 13:04:32.419930 16141 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:32.422092 16119 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
W20260808 13:04:32.423378 15893 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.296s user 0.002s sys 0.013s
I20260808 13:04:32.422282 16119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "55c0037008884da38873ec9e6f80cf41"
format_stamp: "Formatted at 2026-08-08 13:04:29 on dist-test-slave-t3q3"
server_key: "e99375380c2157cbfb754806f731255d"
server_key_iv: "cdc0049191208d9d40b233cf1b1e1d2a"
server_key_version: "encryptionkey@0"
I20260808 13:04:32.423772 16119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:32.425422 15893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:31.117424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.129:42947)
0808 13:04:31.118299 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:31.118338 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:31.119992 (+ 1654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:31.121389 (+ 1397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:31.121404 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:31.122030 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:31.122599 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:31.122618 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:31.124396 (+ 1778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:31.124409 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:31.125686 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:31.125699 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:31.126767 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:31.126775 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:31.127062 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:31.127769 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:31.127793 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:32.424318 (+1296525us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":595,"thread_start_us":352,"threads_started":1}
W20260808 13:04:32.426507 15892 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:31.117348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:42947 (local address 127.1.124.190:38257)
0808 13:04:31.118146 (+ 798us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:31.118162 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:31.118458 (+ 296us) server_negotiation.cc:407] Connection header received
0808 13:04:31.120574 (+ 2116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:31.120586 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:31.120887 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:31.121248 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:31.122807 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:31.124234 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:31.125835 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:31.126512 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:31.126718 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:32.425077 (+1298359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.129:42947: BlockingRecv error: recv got EOF from 127.1.124.129:42947 (error 108)
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":277,"threads_started":1}
W20260808 13:04:32.426604 15890 heartbeater.cc:644] Failed to heartbeat to 127.1.124.190:38257 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.124.190:38257: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Aug 08 13:04:32 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:32 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 13:04:32.443616 15892 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:32.427731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:50783 (local address 127.1.124.190:38257)
0808 13:04:32.427924 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:32.427931 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:32.428707 (+ 776us) server_negotiation.cc:407] Connection header received
0808 13:04:32.428780 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:32.428785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:32.428862 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:32.428959 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:32.430136 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:32.431468 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:32.434199 (+ 2731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:32.435100 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:32.435252 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:32.443366 (+ 8114us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.129:50783: BlockingRecv error: recv got EOF from 127.1.124.129:50783 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 13:04:32.443745 15893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:32.427327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.129:50783)
0808 13:04:32.428399 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:32.428423 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:32.428579 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:32.429176 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:32.429180 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:32.429722 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:32.429952 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:32.429965 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:32.431711 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:32.431720 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:32.433967 (+ 2247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:32.433988 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:32.435587 (+ 1599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:32.435596 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:32.435854 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:32.436808 (+ 954us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:32.436841 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:32.443187 (+ 6346us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":944,"thread_start_us":11593,"threads_started":1}
I20260808 13:04:32.449821 16119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:32.454792 16119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:32.455080 16119 kserver.cc:163] Server-wide thread pool size limit: 3276
Aug 08 13:04:32 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:32 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:32.457481 16119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:32.459142 16119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:32.459264 16119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:32.459390 16119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:32.459479 16119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:32.460014 15893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:32.444747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.129:35031)
0808 13:04:32.445090 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:32.445118 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:32.445284 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:32.445720 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:32.445729 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:32.446391 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:32.446737 (+ 346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:32.446754 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:32.448196 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:32.448206 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:32.450826 (+ 2620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:32.450849 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:32.451931 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:32.451941 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:32.452262 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:32.453128 (+ 866us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:32.453167 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:32.459551 (+ 6384us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182}
W20260808 13:04:32.460139 15892 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:32.444759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:35031 (local address 127.1.124.190:38257)
0808 13:04:32.444898 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:32.444903 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:32.445207 (+ 304us) server_negotiation.cc:407] Connection header received
0808 13:04:32.445437 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:32.445442 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:32.445536 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:32.445633 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:32.446937 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:32.447988 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:32.451025 (+ 3037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:32.451687 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:32.451822 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:32.459780 (+ 7958us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.129:35031: BlockingRecv error: recv got EOF from 127.1.124.129:35031 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 13:04:32.460578 15890 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 13:04:32.502279 16119 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:36891
I20260808 13:04:32.502337 16317 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:36891 every 8 connection(s)
I20260808 13:04:32.503764 16119 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:04:32.509869 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 16119
Aug 08 13:04:32 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:32 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:32.552565 16323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:32.511012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54922 (local address 127.1.124.190:38257)
0808 13:04:32.525194 (+ 14182us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:32.525198 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:32.525213 (+ 15us) server_negotiation.cc:407] Connection header received
0808 13:04:32.525252 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:32.525256 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:32.525307 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:32.525430 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:32.527428 (+ 1998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:32.529677 (+ 2249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:32.532104 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:32.533214 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:32.533419 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:32.537475 (+ 4056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:32.537495 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:32.537534 (+ 39us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:32.537609 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:32.543870 (+ 6261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:32.545368 (+ 1498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:32.545385 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:32.545402 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:32.545772 (+ 370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:32.546490 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:32.546500 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:32.546506 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:32.547558 (+ 1052us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:32.548027 (+ 469us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:32.550890 (+ 2863us) server_negotiation.cc:299] Negotiation successful
0808 13:04:32.551582 (+ 692us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14091,"thread_start_us":13953,"threads_started":1}
W20260808 13:04:32.808944 16114 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.147s user 0.001s sys 0.012s
W20260808 13:04:32.810365 16116 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:31.631970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:57499 (local address 127.1.124.190:38257)
0808 13:04:31.653028 (+ 21058us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:31.653035 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:31.653059 (+ 24us) server_negotiation.cc:407] Connection header received
0808 13:04:31.655540 (+ 2481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:31.655546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:31.655680 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:31.655817 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:31.657486 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:31.658631 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:31.660456 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:31.660876 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:31.661016 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:32.810023 (+1149007us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.130:57499: BlockingRecv error: recv got EOF from 127.1.124.130:57499 (error 108)
Metrics: {"server-negotiator.queue_time_us":20892,"thread_start_us":22977,"threads_started":1}
I20260808 13:04:32.810967 16114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:31.631147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.130:57499)
0808 13:04:31.652213 (+ 21066us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:31.652274 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:31.655190 (+ 2916us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:31.656119 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:31.656131 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:31.656706 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:31.657274 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:31.657291 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:31.658859 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:31.658865 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:31.660309 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:31.660322 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:31.661110 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:31.661119 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:31.661371 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:31.662097 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:31.662126 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:32.809800 (+1147674us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":20706,"thread_start_us":20418,"threads_started":1}
W20260808 13:04:32.811964 16112 heartbeater.cc:644] Failed to heartbeat to 127.1.124.190:38257 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.124.190:38257: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Aug 08 13:04:32 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:32 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:32.827034 16114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:32.812812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.130:53015)
0808 13:04:32.813064 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:32.813081 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:32.813223 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:32.813611 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:32.813616 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:32.814003 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:32.814179 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:32.814187 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:32.815337 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:32.815343 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:32.816687 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:32.816701 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:32.817639 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:32.817649 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:32.817928 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:32.821630 (+ 3702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:32.821668 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:32.826688 (+ 5020us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":92}
W20260808 13:04:32.827199 16116 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:32.812808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:53015 (local address 127.1.124.190:38257)
0808 13:04:32.813003 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:32.813009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:32.813152 (+ 143us) server_negotiation.cc:407] Connection header received
0808 13:04:32.813360 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:32.813364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:32.813425 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:32.813585 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:32.814332 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:32.815193 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:32.816840 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:32.817356 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:32.817548 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:32.826943 (+ 9395us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.130:53015: BlockingRecv error: recv got EOF from 127.1.124.130:53015 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 13:04:32 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:32 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
W20260808 13:04:32.839696 16116 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:32.828670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:59137 (local address 127.1.124.190:38257)
0808 13:04:32.828855 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:32.828867 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:32.828884 (+ 17us) server_negotiation.cc:407] Connection header received
0808 13:04:32.828960 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:32.828965 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:32.829054 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:32.829169 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:32.829961 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:32.831047 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:32.832905 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:32.833921 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:32.834117 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:32.839439 (+ 5322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.130:59137: BlockingRecv error: recv got EOF from 127.1.124.130:59137 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 13:04:32.839689 16114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:32.828107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.130:59137)
0808 13:04:32.828366 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:32.828398 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:32.828586 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:32.829234 (+ 648us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:32.829239 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:32.829570 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:32.829799 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:32.829814 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:32.831245 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:32.831266 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:32.832721 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:32.832734 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:32.834126 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:32.834131 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:32.834265 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:32.834926 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:32.834944 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:32.839237 (+ 4293us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":69}
W20260808 13:04:32.840165 16112 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 13:04:33 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:33 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:33.471946 16326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:33.461295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.129:56393)
0808 13:04:33.461863 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:33.461886 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:33.462038 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:33.462449 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:33.462454 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:33.462926 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:33.463088 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:33.463096 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:33.464403 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:33.464408 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:33.465372 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:33.465383 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:33.466139 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:33.466148 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:33.466312 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:33.467040 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:33.467077 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:33.471599 (+ 4522us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":324,"threads_started":1}
W20260808 13:04:33.472141 16327 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:33.461404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:56393 (local address 127.1.124.190:38257)
0808 13:04:33.462154 (+ 750us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:33.462160 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:33.462182 (+ 22us) server_negotiation.cc:407] Connection header received
0808 13:04:33.462224 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:33.462227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:33.462284 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:33.462396 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:33.463236 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:33.464265 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:33.465525 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:33.465950 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:33.466076 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:33.471813 (+ 5737us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.129:56393: BlockingRecv error: recv got EOF from 127.1.124.129:56393 (error 108)
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":463,"threads_started":1}
W20260808 13:04:33.640974 16322 sasl_common.cc:323] Time spent calling sasl_client_start(): real 1.118s user 0.008s sys 0.009s
W20260808 13:04:33.642360 15892 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:32.506830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:45185 (local address 127.1.124.190:38257)
0808 13:04:32.507062 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:32.507067 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:32.509201 (+ 2134us) server_negotiation.cc:407] Connection header received
0808 13:04:32.512173 (+ 2972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:32.512182 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:32.512298 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:32.512390 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:32.515309 (+ 2919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:32.516874 (+ 1565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:32.520104 (+ 3230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:32.520987 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:32.521113 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:33.642166 (+1121053us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.131:45185: BlockingRecv error: recv got EOF from 127.1.124.131:45185 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 13:04:33.643186 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:32.507936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.131:45185)
0808 13:04:32.509025 (+ 1089us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:32.509067 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:32.511709 (+ 2642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:32.513056 (+ 1347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:32.513072 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:32.514041 (+ 969us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:32.515048 (+ 1007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:32.515079 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:32.517162 (+ 2083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:32.517170 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:32.519842 (+ 2672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:32.519870 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:32.521540 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:32.521550 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:32.521934 (+ 384us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:32.522940 (+ 1006us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:32.522995 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:33.642043 (+1119048us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":772,"thread_start_us":450,"threads_started":1}
W20260808 13:04:33.644227 16318 heartbeater.cc:644] Failed to heartbeat to 127.1.124.190:38257 (0 consecutive failures): Not authorized: Failed to ping master at 127.1.124.190:38257: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Aug 08 13:04:33 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:33 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:33.653930 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:33.644923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.131:35959)
0808 13:04:33.645138 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:33.645162 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:33.645284 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:33.645610 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:33.645613 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:33.645995 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:33.646231 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:33.646247 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:33.647372 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:33.647377 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:33.648459 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:33.648472 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:33.649197 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:33.649202 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:33.649324 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:33.649924 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:33.649943 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:33.653668 (+ 3725us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":70}
W20260808 13:04:33.654184 15892 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:33.645079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:35959 (local address 127.1.124.190:38257)
0808 13:04:33.645303 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:33.645308 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:33.645349 (+ 41us) server_negotiation.cc:407] Connection header received
0808 13:04:33.645396 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:33.645399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:33.645463 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:33.645536 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:33.646390 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:33.647221 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:33.648617 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:33.649040 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:33.649138 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:33.653922 (+ 4784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.131:35959: BlockingRecv error: recv got EOF from 127.1.124.131:35959 (error 108)
Metrics: {"server-negotiator.queue_time_us":123}
Aug 08 13:04:33 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:33 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:33.662883 16322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:33.654654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.131:36779)
0808 13:04:33.654803 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:33.654826 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:33.654951 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:33.655308 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:33.655312 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:33.655634 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:33.655783 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:33.655792 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:33.656831 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:33.656836 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:33.657787 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:33.657799 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:33.658499 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:33.658505 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:33.658690 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:33.659278 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:33.659295 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:33.662607 (+ 3312us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 13:04:33.663064 15892 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:33.654836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:36779 (local address 127.1.124.190:38257)
0808 13:04:33.655031 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:33.655039 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:33.655057 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:04:33.655102 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:33.655106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:33.655166 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:33.655267 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:33.655918 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:33.656693 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:33.657930 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:33.658342 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:33.658468 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:33.662827 (+ 4359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.131:36779: BlockingRecv error: recv got EOF from 127.1.124.131:36779 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 13:04:33.663386 16318 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 13:04:33 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:33 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:33.851063 16328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:33.840789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.130:40863)
0808 13:04:33.841407 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:33.841423 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:33.841619 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:33.842009 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:33.842014 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:33.842486 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:33.842651 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:33.842661 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:33.843822 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:33.843827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:33.845013 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:33.845033 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:33.846071 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:33.846078 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:33.846307 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:33.846932 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:33.846952 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:33.850764 (+ 3812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":359,"threads_started":1}
W20260808 13:04:33.851337 15892 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:33.840894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:40863 (local address 127.1.124.190:38257)
0808 13:04:33.841224 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:33.841231 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:33.841523 (+ 292us) server_negotiation.cc:407] Connection header received
0808 13:04:33.841787 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:33.841791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:33.841855 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:33.841964 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:33.842884 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:33.843675 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:33.845220 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:33.845860 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:33.845972 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:33.851021 (+ 5049us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.130:40863: BlockingRecv error: recv got EOF from 127.1.124.130:40863 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 08 13:04:34 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:34 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:34.483070 16330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:34.473106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.129:48723)
0808 13:04:34.473885 (+ 779us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:34.473901 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:34.474026 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:34.474343 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:34.474346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:34.474759 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:34.474948 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:34.474957 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:34.476028 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:34.476032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:34.477063 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:34.477075 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:34.477973 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:34.477980 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:34.478202 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:34.478889 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:34.478908 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:34.482647 (+ 3739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":690,"thread_start_us":590,"threads_started":1}
W20260808 13:04:34.483129 16331 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:34.473105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:48723 (local address 127.1.124.190:38257)
0808 13:04:34.473613 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:34.473618 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:34.473987 (+ 369us) server_negotiation.cc:407] Connection header received
0808 13:04:34.474147 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:34.474150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:34.474213 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:34.474298 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:34.475082 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:34.475868 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:34.477215 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:34.477796 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:34.477952 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:34.482840 (+ 4888us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.129:48723: BlockingRecv error: recv got EOF from 127.1.124.129:48723 (error 108)
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":240,"threads_started":1}
Aug 08 13:04:34 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:34 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:34.673583 16338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:34.663966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.131:46683)
0808 13:04:34.664587 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:34.664603 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:34.664760 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:34.665113 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:34.665118 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:34.665618 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:34.665774 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:34.665783 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:34.666816 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:34.666823 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:34.667794 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:34.667805 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:34.668620 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:34.668627 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:34.668810 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:34.669377 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:34.669394 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:34.673339 (+ 3945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":356,"threads_started":1}
W20260808 13:04:34.673799 16331 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:34.664210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:46683 (local address 127.1.124.190:38257)
0808 13:04:34.664426 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:34.664433 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:34.664689 (+ 256us) server_negotiation.cc:407] Connection header received
0808 13:04:34.664894 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:34.664900 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:34.664971 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:34.665085 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:34.665920 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:34.666686 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:34.667940 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:34.668424 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:34.668564 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:34.673554 (+ 4990us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.131:46683: BlockingRecv error: recv got EOF from 127.1.124.131:46683 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 08 13:04:34 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:34 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:34.861577 16339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:34.852260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.130:42369)
0808 13:04:34.852911 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:34.852929 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:34.853092 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:34.853428 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:34.853431 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:34.853834 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:34.854098 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:34.854110 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:34.855263 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:34.855269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:34.856269 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:34.856281 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:34.856912 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:34.856916 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:34.857033 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:34.857605 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:34.857626 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:34.861184 (+ 3558us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":415,"threads_started":1}
W20260808 13:04:34.861636 16331 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:34.852291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:42369 (local address 127.1.124.190:38257)
0808 13:04:34.852495 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:34.852504 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:34.853028 (+ 524us) server_negotiation.cc:407] Connection header received
0808 13:04:34.853230 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:34.853235 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:34.853305 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:34.853386 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:34.854332 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:34.855108 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:34.856390 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:34.856785 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:34.856876 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:34.861382 (+ 4506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.130:42369: BlockingRecv error: recv got EOF from 127.1.124.130:42369 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 13:04:35 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:35 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:35.493032 16342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:35.484202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.129:34659)
0808 13:04:35.484890 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:35.484914 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:35.485098 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:35.485475 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:35.485479 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:35.486008 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:35.486237 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:35.486247 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:35.487233 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:35.487237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:35.488170 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:35.488181 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:35.488822 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:35.488826 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:35.488950 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:35.489517 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:35.489534 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:35.492785 (+ 3251us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":300,"threads_started":1}
W20260808 13:04:35.493260 16343 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:35.484457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:34659 (local address 127.1.124.190:38257)
0808 13:04:35.485095 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:35.485119 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:35.485142 (+ 23us) server_negotiation.cc:407] Connection header received
0808 13:04:35.485210 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:35.485217 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:35.485307 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:35.485427 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:35.486374 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:35.487120 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:35.488286 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:35.488693 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:35.488783 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:35.493001 (+ 4218us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.129:34659: BlockingRecv error: recv got EOF from 127.1.124.129:34659 (error 108)
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":412,"threads_started":1}
Aug 08 13:04:35 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:35 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:35.684139 16344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:35.674618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.131:38359)
0808 13:04:35.675180 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:35.675197 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:35.675316 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:35.675636 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:35.675640 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:35.676044 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:35.676303 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:35.676312 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:35.677607 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:35.677613 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:35.678919 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:35.678935 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:35.679588 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:35.679593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:35.679728 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:35.680350 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:35.680369 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:35.683806 (+ 3437us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":364,"threads_started":1}
W20260808 13:04:35.684324 16343 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:35.674688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:38359 (local address 127.1.124.190:38257)
0808 13:04:35.674865 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:35.674870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:35.675272 (+ 402us) server_negotiation.cc:407] Connection header received
0808 13:04:35.675434 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:35.675438 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:35.675500 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:35.675582 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:35.676441 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:35.677427 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:35.679051 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:35.679435 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:35.679528 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:35.684006 (+ 4478us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.131:38359: BlockingRecv error: recv got EOF from 127.1.124.131:38359 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 13:04:35 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:35 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:35.871572 16348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:35.862674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.130:43405)
0808 13:04:35.863173 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:35.863189 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:35.863311 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:35.863616 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:35.863619 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:35.863973 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:35.864173 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:35.864181 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:35.865143 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:35.865146 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:35.866030 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:35.866040 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:35.866706 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:35.866710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:35.866825 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:35.867461 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:35.867477 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:35.871203 (+ 3726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":264,"threads_started":1}
W20260808 13:04:35.871672 16343 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:35.862719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:43405 (local address 127.1.124.190:38257)
0808 13:04:35.862990 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:35.862998 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:35.863273 (+ 275us) server_negotiation.cc:407] Connection header received
0808 13:04:35.863413 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:35.863416 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:35.863490 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:35.863582 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:35.864291 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:35.865024 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:35.866153 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:35.866582 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:35.866675 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:35.871397 (+ 4722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.130:43405: BlockingRecv error: recv got EOF from 127.1.124.130:43405 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Aug 08 13:04:36 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:36 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:36.503388 16351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:36.494007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.129:54359)
0808 13:04:36.494618 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:36.494635 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:36.494750 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:36.495061 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:36.495064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:36.495457 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:36.495625 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:36.495633 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:36.496855 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:36.496859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:36.497857 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:36.497868 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:36.498813 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:36.498822 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:36.499030 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:36.499671 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:36.499695 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:36.503147 (+ 3452us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":370,"threads_started":1}
W20260808 13:04:36.503700 16352 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:36.494186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:54359 (local address 127.1.124.190:38257)
0808 13:04:36.494694 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:36.494698 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:36.494716 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:04:36.494851 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:36.494855 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:36.494913 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:36.495012 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:36.495763 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:36.496727 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:36.498031 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:36.498629 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:36.498778 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:36.503401 (+ 4623us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.129:54359: BlockingRecv error: recv got EOF from 127.1.124.129:54359 (error 108)
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":286,"threads_started":1}
Aug 08 13:04:36 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:36 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:36.695029 16353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:36.685230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.131:58391)
0808 13:04:36.685795 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:36.685813 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:36.685942 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:36.686394 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:36.686397 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:36.686809 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:36.686996 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:36.687008 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:36.688001 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:36.688006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:36.689153 (+ 1147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:36.689168 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:36.689874 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:36.689878 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:36.690015 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:36.690663 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:36.690681 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:36.694797 (+ 4116us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":316,"threads_started":1}
W20260808 13:04:36.695264 16352 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:36.685304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:58391 (local address 127.1.124.190:38257)
0808 13:04:36.685566 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:36.685573 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:36.685911 (+ 338us) server_negotiation.cc:407] Connection header received
0808 13:04:36.686147 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:36.686151 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:36.686219 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:36.686308 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:36.687140 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:36.687865 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:36.689301 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:36.689725 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:36.689833 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:36.695043 (+ 5210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.131:58391: BlockingRecv error: recv got EOF from 127.1.124.131:58391 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 08 13:04:36 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:36 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:36.882284 16354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:36.872682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.130:58129)
0808 13:04:36.873234 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:36.873250 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:36.873372 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:36.873719 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:36.873723 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:36.874164 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:36.874352 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:36.874363 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:36.875426 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:36.875431 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:36.876830 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:36.876843 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:36.877670 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:36.877675 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:36.877805 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:36.878410 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:36.878428 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:36.882029 (+ 3601us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":316,"threads_started":1}
W20260808 13:04:36.882596 16352 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:36.872796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:58129 (local address 127.1.124.190:38257)
0808 13:04:36.872974 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:36.872979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:36.873324 (+ 345us) server_negotiation.cc:407] Connection header received
0808 13:04:36.873471 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:36.873475 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:36.873548 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:36.873634 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:36.874490 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:36.875279 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:36.877001 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:36.877526 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:36.877623 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:36.882234 (+ 4611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.130:58129: BlockingRecv error: recv got EOF from 127.1.124.130:58129 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 13:04:37 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:37 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:37.513811 16355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:37.504598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.129:60979)
0808 13:04:37.505231 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:37.505248 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:37.505371 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:37.505673 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:37.505676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:37.506109 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:37.506257 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:37.506266 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:37.507350 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:37.507357 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:37.508514 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:37.508527 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:37.509215 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:37.509221 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:37.509403 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:37.509989 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:37.510043 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:37.513484 (+ 3441us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":337,"threads_started":1}
W20260808 13:04:37.513980 16359 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:37.504665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:60979 (local address 127.1.124.190:38257)
0808 13:04:37.505234 (+ 569us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:37.505240 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:37.505325 (+ 85us) server_negotiation.cc:407] Connection header received
0808 13:04:37.505470 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:37.505474 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:37.505536 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:37.505629 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:37.506423 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:37.507208 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:37.508649 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:37.509068 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:37.509201 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:37.513705 (+ 4504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.129:60979: BlockingRecv error: recv got EOF from 127.1.124.129:60979 (error 108)
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":295,"threads_started":1}
Aug 08 13:04:37 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:37 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:37.705840 16360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:37.696148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.131:44735)
0808 13:04:37.696715 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:37.696731 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:37.696857 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:37.697190 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:37.697193 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:37.697564 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:37.697730 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:37.697738 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:37.698863 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:37.698869 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:37.700396 (+ 1527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:37.700411 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:37.701247 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:37.701252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:37.701380 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:37.702034 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:37.702053 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:37.705567 (+ 3514us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":320,"threads_started":1}
W20260808 13:04:37.706130 16359 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:37.696225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:44735 (local address 127.1.124.190:38257)
0808 13:04:37.696398 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:37.696403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:37.696804 (+ 401us) server_negotiation.cc:407] Connection header received
0808 13:04:37.696981 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:37.696985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:37.697054 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:37.697132 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:37.697891 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:37.698717 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:37.700602 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:37.701094 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:37.701204 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:37.705835 (+ 4631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.131:44735: BlockingRecv error: recv got EOF from 127.1.124.131:44735 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 13:04:37 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:37 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:37.896162 16361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:37.883293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.130:36465)
0808 13:04:37.883941 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:37.883958 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:37.884144 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:37.884509 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:37.884513 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:37.885066 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:37.885230 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:37.885240 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:37.886421 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:37.886428 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:37.887386 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:37.887397 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:37.888148 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:37.888154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:37.888341 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:37.892332 (+ 3991us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:37.892352 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:37.895804 (+ 3452us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":393,"threads_started":1}
W20260808 13:04:37.896360 16359 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:37.883517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:36465 (local address 127.1.124.190:38257)
0808 13:04:37.883783 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:37.883791 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:37.884051 (+ 260us) server_negotiation.cc:407] Connection header received
0808 13:04:37.884283 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:37.884288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:37.884364 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:37.884470 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:37.885380 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:37.886283 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:37.887528 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:37.887962 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:37.888112 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:37.896046 (+ 7934us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.130:36465: BlockingRecv error: recv got EOF from 127.1.124.130:36465 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 08 13:04:38 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:38 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:38.524923 16362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:38.514899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.129:44957)
0808 13:04:38.515745 (+ 846us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:38.515777 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:38.515974 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:38.516389 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:38.516393 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:38.516832 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:38.516999 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:38.517008 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:38.518049 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:38.518052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:38.518963 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:38.518974 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:38.519636 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:38.519640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:38.519766 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:38.520359 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:38.520375 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:38.524564 (+ 4189us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":585,"thread_start_us":356,"threads_started":1}
W20260808 13:04:38.525090 16363 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:38.515071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:44957 (local address 127.1.124.190:38257)
0808 13:04:38.515946 (+ 875us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:38.515955 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:38.515999 (+ 44us) server_negotiation.cc:407] Connection header received
0808 13:04:38.516120 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:38.516126 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:38.516224 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:38.516353 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:38.517167 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:38.517912 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:38.519096 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:38.519486 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:38.519590 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:38.524812 (+ 5222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.129:44957: BlockingRecv error: recv got EOF from 127.1.124.129:44957 (error 108)
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":571,"threads_started":1}
Aug 08 13:04:38 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:38 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:38.716322 16364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:38.706822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.131:57481)
0808 13:04:38.707575 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:38.707591 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:38.707750 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:38.708126 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:38.708133 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:38.708593 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:38.708745 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:38.708753 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:38.709798 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:38.709803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:38.710745 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:38.710757 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:38.711434 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:38.711440 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:38.711617 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:38.712388 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:38.712407 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:38.715999 (+ 3592us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":651,"thread_start_us":516,"threads_started":1}
W20260808 13:04:38.716480 16363 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:38.706970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:57481 (local address 127.1.124.190:38257)
0808 13:04:38.707165 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:38.707174 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:38.707682 (+ 508us) server_negotiation.cc:407] Connection header received
0808 13:04:38.707879 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:38.707889 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:38.707952 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:38.708053 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:38.708890 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:38.709649 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:38.710888 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:38.711294 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:38.711422 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:38.716285 (+ 4863us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.131:57481: BlockingRecv error: recv got EOF from 127.1.124.131:57481 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 13:04:38 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:38 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:38.906572 16365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:38.897293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.130:52235)
0808 13:04:38.897818 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:38.897841 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:38.898000 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:38.898406 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:38.898410 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:38.898935 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:38.899142 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:38.899155 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:38.900502 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:38.900506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:38.901448 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:38.901459 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:38.902120 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:38.902125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:38.902258 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:38.902894 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:38.902912 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:38.906340 (+ 3428us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":269,"threads_started":1}
W20260808 13:04:38.907069 16363 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:38.897384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:52235 (local address 127.1.124.190:38257)
0808 13:04:38.897635 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:38.897642 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:38.897938 (+ 296us) server_negotiation.cc:407] Connection header received
0808 13:04:38.898154 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:38.898164 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:38.898251 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:38.898332 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:38.899354 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:38.900361 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:38.901570 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:38.901970 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:38.902076 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:38.906580 (+ 4504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.130:52235: BlockingRecv error: recv got EOF from 127.1.124.130:52235 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 08 13:04:39 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:39 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:39.535406 16372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:39.525901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.129:41143)
0808 13:04:39.526544 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:39.526562 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:39.526675 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:39.527025 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:39.527029 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:39.527433 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:39.527633 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:39.527642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:39.528787 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:39.528791 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:39.529846 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:39.529859 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:39.530576 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:39.530580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:39.530700 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:39.531354 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:39.531371 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:39.535149 (+ 3778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":506,"thread_start_us":261,"threads_started":1}
W20260808 13:04:39.535692 16375 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:39.526079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:41143 (local address 127.1.124.190:38257)
0808 13:04:39.526586 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:39.526591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:39.526609 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:04:39.526781 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:39.526785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:39.526857 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:39.526948 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:39.527763 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:39.528650 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:39.529971 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:39.530429 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:39.530526 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:39.535353 (+ 4827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.129:41143: BlockingRecv error: recv got EOF from 127.1.124.129:41143 (error 108)
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":266,"threads_started":1}
Aug 08 13:04:39 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:39 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:39.726871 16378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:39.717452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.131:38717)
0808 13:04:39.717951 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:39.717967 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:39.718156 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:39.718514 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:39.718519 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:39.718974 (+ 455us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:39.719150 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:39.719159 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:39.720266 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:39.720272 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:39.721217 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:39.721228 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:39.721941 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:39.721947 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:39.722137 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:39.722763 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:39.722781 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:39.726511 (+ 3730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":254,"threads_started":1}
W20260808 13:04:39.727180 16375 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:39.717550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:38717 (local address 127.1.124.190:38257)
0808 13:04:39.717784 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:39.717792 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:39.718054 (+ 262us) server_negotiation.cc:407] Connection header received
0808 13:04:39.718301 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:39.718306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:39.718379 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:39.718480 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:39.719305 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:39.720129 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:39.721353 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:39.721785 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:39.721922 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:39.726728 (+ 4806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.131:38717: BlockingRecv error: recv got EOF from 127.1.124.131:38717 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 08 13:04:39 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:39 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:39.917059 16381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:39.907605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.130:39505)
0808 13:04:39.908302 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:39.908323 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:39.908461 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:39.908852 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:39.908856 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:39.909304 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:39.909473 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:39.909481 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:39.910506 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:39.910511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:39.911463 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:39.911473 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:39.912246 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:39.912251 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:39.912438 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:39.913008 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:39.913024 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:39.916771 (+ 3747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":296,"threads_started":1}
W20260808 13:04:39.917372 16375 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:39.907706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:39505 (local address 127.1.124.190:38257)
0808 13:04:39.908039 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:39.908274 (+ 235us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:39.908412 (+ 138us) server_negotiation.cc:407] Connection header received
0808 13:04:39.908612 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:39.908618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:39.908700 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:39.908802 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:39.909618 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:39.910368 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:39.911603 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:39.912054 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:39.912233 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:39.917044 (+ 4811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.130:39505: BlockingRecv error: recv got EOF from 127.1.124.130:39505 (error 108)
Metrics: {"server-negotiator.queue_time_us":137}
Aug 08 13:04:40 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:40 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:40.545899 16382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:40.536512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.129:56885)
0808 13:04:40.537104 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:40.537120 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:40.537261 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:40.537557 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:40.537560 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:40.537967 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:40.538145 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:40.538154 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:40.539174 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:40.539177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:40.540161 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:40.540179 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:40.540926 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:40.540931 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:40.541042 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:40.541685 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:40.541702 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:40.545534 (+ 3832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":304,"threads_started":1}
W20260808 13:04:40.546054 16383 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:40.536557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:56885 (local address 127.1.124.190:38257)
0808 13:04:40.537066 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:40.537072 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:40.537196 (+ 124us) server_negotiation.cc:407] Connection header received
0808 13:04:40.537348 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:40.537351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:40.537422 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:40.537516 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:40.538285 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:40.539043 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:40.540312 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:40.540772 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:40.540869 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:40.545779 (+ 4910us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.129:56885: BlockingRecv error: recv got EOF from 127.1.124.129:56885 (error 108)
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":300,"threads_started":1}
Aug 08 13:04:40 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:40 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:40.742331 16384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:40.727792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.131:46219)
0808 13:04:40.728591 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:40.728614 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:40.728837 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:40.729486 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:40.729491 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:40.730120 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:40.730398 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:40.730416 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:40.732193 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:40.732199 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:40.733928 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:40.733945 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:40.735215 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:40.735222 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:40.735411 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:40.736137 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:40.736165 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:40.742076 (+ 5911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":3329,"threads_started":1}
W20260808 13:04:40.742468 16383 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:40.728000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:46219 (local address 127.1.124.190:38257)
0808 13:04:40.728306 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:40.728315 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:40.728700 (+ 385us) server_negotiation.cc:407] Connection header received
0808 13:04:40.729012 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:40.729019 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:40.729127 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:40.729209 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:40.730604 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:40.731945 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:40.734223 (+ 2278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:40.735023 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:40.735165 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:40.742255 (+ 7090us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.131:46219: BlockingRecv error: recv got EOF from 127.1.124.131:46219 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 08 13:04:40 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:40 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:40.927903 16385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:40.918086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.130:58801)
0808 13:04:40.918616 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:40.918633 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:40.918823 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:40.919177 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:40.919180 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:40.919671 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:40.919840 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:40.919848 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:40.921171 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:40.921176 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:40.922465 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:40.922481 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:40.923427 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:40.923433 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:40.923554 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:40.924174 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:40.924191 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:40.927639 (+ 3448us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":272,"threads_started":1}
W20260808 13:04:40.928062 16383 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:40.918163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:58801 (local address 127.1.124.190:38257)
0808 13:04:40.918407 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:40.918414 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:40.918738 (+ 324us) server_negotiation.cc:407] Connection header received
0808 13:04:40.918959 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:40.918963 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:40.919025 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:40.919109 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:40.919971 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:40.920988 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:40.922612 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:40.923214 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:40.923337 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:40.927847 (+ 4510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.130:58801: BlockingRecv error: recv got EOF from 127.1.124.130:58801 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 13:04:41 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:41 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:41.556797 16386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:41.547003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.129:54811)
0808 13:04:41.547586 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:41.547604 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:41.547725 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:41.548104 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:41.548107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:41.548524 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:41.548673 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:41.548682 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:41.550005 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:41.550009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:41.551248 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:41.551261 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:41.552114 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:41.552123 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:41.552265 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:41.552921 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:41.552938 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:41.556561 (+ 3623us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":322,"threads_started":1}
W20260808 13:04:41.557067 16387 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:41.547068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:54811 (local address 127.1.124.190:38257)
0808 13:04:41.547627 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:41.547632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:41.547673 (+ 41us) server_negotiation.cc:407] Connection header received
0808 13:04:41.547830 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:41.547833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:41.547895 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:41.547984 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:41.548827 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:41.549855 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:41.551375 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:41.551893 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:41.551987 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:41.556739 (+ 4752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.129:54811: BlockingRecv error: recv got EOF from 127.1.124.129:54811 (error 108)
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":290,"threads_started":1}
Aug 08 13:04:41 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:41 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:41.753353 16388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:41.743614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.131:48211)
0808 13:04:41.744307 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:41.744330 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:41.744481 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:41.744928 (+ 447us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:41.744933 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:41.745553 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:41.745778 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:41.745793 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:41.747048 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:41.747052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:41.748017 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:41.748036 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:41.748750 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:41.748755 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:41.748872 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:41.749480 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:41.749498 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:41.753004 (+ 3506us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":309,"threads_started":1}
W20260808 13:04:41.753475 16387 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:41.743709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:48211 (local address 127.1.124.190:38257)
0808 13:04:41.743963 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:41.743970 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:41.744428 (+ 458us) server_negotiation.cc:407] Connection header received
0808 13:04:41.744681 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:41.744687 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:41.744774 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:41.744870 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:41.745938 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:41.746909 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:41.748192 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:41.748598 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:41.748697 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:41.753241 (+ 4544us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.131:48211: BlockingRecv error: recv got EOF from 127.1.124.131:48211 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 08 13:04:41 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:41 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:41.938923 16389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:41.928950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.130:59037)
0808 13:04:41.929540 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:41.929556 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:41.929737 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:41.930101 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:41.930105 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:41.930624 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:41.930786 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:41.930794 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:41.931835 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:41.931840 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:41.932865 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:41.932876 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:41.933804 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:41.933810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:41.934006 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:41.934599 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:41.934617 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:41.938586 (+ 3969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":307,"threads_started":1}
W20260808 13:04:41.939079 16387 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:41.929016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:59037 (local address 127.1.124.190:38257)
0808 13:04:41.929324 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:41.929331 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:41.929637 (+ 306us) server_negotiation.cc:407] Connection header received
0808 13:04:41.929867 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:41.929874 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:41.929963 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:41.930069 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:41.930926 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:41.931695 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:41.933166 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:41.933629 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:41.933791 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:41.938759 (+ 4968us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.130:59037: BlockingRecv error: recv got EOF from 127.1.124.130:59037 (error 108)
Metrics: {"server-negotiator.queue_time_us":106}
Aug 08 13:04:42 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:42 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:42.567948 16391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:42.557807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.129:48165)
0808 13:04:42.558709 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:42.558728 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:42.558904 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:42.559346 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:42.559351 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:42.559823 (+ 472us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:42.560002 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:42.560012 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:42.561201 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:42.561207 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:42.562306 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:42.562319 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:42.563170 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:42.563177 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:42.563360 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:42.564104 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:42.564139 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:42.567700 (+ 3561us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":757,"thread_start_us":565,"threads_started":1}
W20260808 13:04:42.568288 16392 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:42.557858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:48165 (local address 127.1.124.190:38257)
0808 13:04:42.558798 (+ 940us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:42.558809 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:42.558826 (+ 17us) server_negotiation.cc:407] Connection header received
0808 13:04:42.559100 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:42.559104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:42.559184 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:42.559311 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:42.560199 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:42.561029 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:42.562469 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:42.562956 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:42.563130 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:42.567936 (+ 4806us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.129:48165: BlockingRecv error: recv got EOF from 127.1.124.129:48165 (error 108)
Metrics: {"server-negotiator.queue_time_us":822,"thread_start_us":671,"threads_started":1}
Aug 08 13:04:42 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:42 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:42.764109 16394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:42.754526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.131:55073)
0808 13:04:42.755075 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:42.755091 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:42.755217 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:42.755641 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:42.755644 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:42.756157 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:42.756313 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:42.756322 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:42.757402 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:42.757406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:42.758362 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:42.758374 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:42.759084 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:42.759088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:42.759220 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:42.759840 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:42.759857 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:42.763711 (+ 3854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":298,"threads_started":1}
W20260808 13:04:42.764325 16392 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:42.754581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:55073 (local address 127.1.124.190:38257)
0808 13:04:42.754772 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:42.754777 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:42.755172 (+ 395us) server_negotiation.cc:407] Connection header received
0808 13:04:42.755428 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:42.755440 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:42.755505 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:42.755590 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:42.756458 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:42.757264 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:42.758503 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:42.758916 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:42.759008 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:42.764005 (+ 4997us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.131:55073: BlockingRecv error: recv got EOF from 127.1.124.131:55073 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 08 13:04:42 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:42 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:42.953490 16395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:42.939846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.130:54619)
0808 13:04:42.940403 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:42.940420 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:42.940578 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:42.940952 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:42.940957 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:42.941362 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:42.941524 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:42.941532 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:42.942600 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:42.942608 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:42.943734 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:42.943752 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:42.944681 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:42.944688 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:42.944883 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:42.949520 (+ 4637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:42.949540 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:42.953249 (+ 3709us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":311,"threads_started":1}
W20260808 13:04:42.953738 16392 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:42.939927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:54619 (local address 127.1.124.190:38257)
0808 13:04:42.940183 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:42.940191 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:42.940506 (+ 315us) server_negotiation.cc:407] Connection header received
0808 13:04:42.940716 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:42.940721 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:42.940810 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:42.940915 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:42.941676 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:42.942454 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:42.943880 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:42.944502 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:42.944632 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:42.953472 (+ 8840us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.130:54619: BlockingRecv error: recv got EOF from 127.1.124.130:54619 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 08 13:04:43 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:43 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:43.578398 16396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:43.568867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.129:60487)
0808 13:04:43.569368 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:43.569385 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:43.569508 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:43.569971 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:43.569974 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:43.570448 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:43.570646 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:43.570660 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:43.571695 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:43.571699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:43.572692 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:43.572705 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:43.573559 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:43.573565 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:43.573754 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:43.574446 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:43.574465 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:43.578180 (+ 3715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":287,"threads_started":1}
W20260808 13:04:43.578776 16398 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:43.569130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:60487 (local address 127.1.124.190:38257)
0808 13:04:43.569690 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:43.569694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:43.569709 (+ 15us) server_negotiation.cc:407] Connection header received
0808 13:04:43.569752 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:43.569756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:43.569822 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:43.569913 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:43.570788 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:43.571532 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:43.572850 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:43.573382 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:43.573535 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:43.578412 (+ 4877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.129:60487: BlockingRecv error: recv got EOF from 127.1.124.129:60487 (error 108)
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":310,"threads_started":1}
Aug 08 13:04:43 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:43 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:43.774828 16402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:43.765111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.131:33745)
0808 13:04:43.765684 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:43.765701 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:43.765827 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:43.766262 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:43.766265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:43.766814 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:43.767018 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:43.767034 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:43.768377 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:43.768381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:43.769393 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:43.769407 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:43.770069 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:43.770074 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:43.770204 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:43.770859 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:43.770897 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:43.774471 (+ 3574us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":313,"threads_started":1}
W20260808 13:04:43.774883 16398 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:43.765190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:33745 (local address 127.1.124.190:38257)
0808 13:04:43.765376 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:43.765381 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:43.765794 (+ 413us) server_negotiation.cc:407] Connection header received
0808 13:04:43.765961 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:43.765969 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:43.766055 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:43.766186 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:43.767216 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:43.768244 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:43.769534 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:43.769934 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:43.770027 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:43.774671 (+ 4644us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.131:33745: BlockingRecv error: recv got EOF from 127.1.124.131:33745 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 13:04:43 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:43 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:43.964133 16405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:43.954458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.130:42601)
0808 13:04:43.954972 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:43.954989 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:43.955164 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:43.955529 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:43.955534 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:43.955980 (+ 446us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:43.956220 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:43.956229 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:43.957273 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:43.957280 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:43.958196 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:43.958207 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:43.958952 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:43.958958 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:43.959134 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:43.959720 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:43.959737 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:43.963776 (+ 4039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":285,"threads_started":1}
W20260808 13:04:43.964290 16398 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:43.954523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:42601 (local address 127.1.124.190:38257)
0808 13:04:43.954805 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:43.954815 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:43.955085 (+ 270us) server_negotiation.cc:407] Connection header received
0808 13:04:43.955290 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:43.955296 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:43.955380 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:43.955486 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:43.956356 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:43.957139 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:43.958350 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:43.958783 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:43.958915 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:43.963983 (+ 5068us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.130:42601: BlockingRecv error: recv got EOF from 127.1.124.130:42601 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
Aug 08 13:04:44 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:44 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:44.588868 16408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:44.579429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.129:48231)
0808 13:04:44.580012 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:44.580028 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:44.580217 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:44.580592 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:44.580595 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:44.581001 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:44.581191 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:44.581200 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:44.582243 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:44.582246 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:44.583180 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:44.583191 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:44.583973 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:44.583978 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:44.584168 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:44.584894 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:44.584912 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:44.588493 (+ 3581us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":324,"threads_started":1}
W20260808 13:04:44.588949 16409 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:44.579481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:48231 (local address 127.1.124.190:38257)
0808 13:04:44.580120 (+ 639us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:44.580130 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:44.580148 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:04:44.580372 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:44.580376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:44.580449 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:44.580554 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:44.581349 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:44.582110 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:44.583311 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:44.583784 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:44.583882 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:44.588699 (+ 4817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.129:48231: BlockingRecv error: recv got EOF from 127.1.124.129:48231 (error 108)
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":326,"threads_started":1}
Aug 08 13:04:44 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:44 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:44.785668 16411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:44.775977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.131:59289)
0808 13:04:44.776579 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:44.776596 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:44.776714 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:44.777100 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:44.777103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:44.777529 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:44.777716 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:44.777724 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:44.778921 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:44.778924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:44.779833 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:44.779844 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:44.780581 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:44.780585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:44.780707 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:44.781350 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:44.781368 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:44.785441 (+ 4073us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":357,"threads_started":1}
W20260808 13:04:44.785816 16409 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:44.776111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:59289 (local address 127.1.124.190:38257)
0808 13:04:44.776298 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:44.776303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:44.776674 (+ 371us) server_negotiation.cc:407] Connection header received
0808 13:04:44.776886 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:44.776890 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:44.776955 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:44.777090 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:44.777834 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:44.778796 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:44.779951 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:44.780440 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:44.780535 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:44.785611 (+ 5076us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.131:59289: BlockingRecv error: recv got EOF from 127.1.124.131:59289 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 08 13:04:44 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:44 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:44.974972 16413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:44.965384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.130:53861)
0808 13:04:44.966081 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:44.966100 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:44.966227 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:44.966623 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:44.966628 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:44.967298 (+ 670us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:44.967560 (+ 262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:44.967574 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:44.968726 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:44.968730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:44.969674 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:44.969686 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:44.970384 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:44.970388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:44.970538 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:44.971159 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:44.971176 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:44.974709 (+ 3533us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":527,"thread_start_us":361,"threads_started":1}
W20260808 13:04:44.975288 16409 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:44.965460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:53861 (local address 127.1.124.190:38257)
0808 13:04:44.965736 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:44.965746 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:44.966226 (+ 480us) server_negotiation.cc:407] Connection header received
0808 13:04:44.966334 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:44.966345 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:44.966446 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:44.966550 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:44.967733 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:44.968581 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:44.969799 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:44.970225 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:44.970318 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:44.974930 (+ 4612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.130:53861: BlockingRecv error: recv got EOF from 127.1.124.130:53861 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 08 13:04:45 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:45 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:45.600287 16414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:45.590041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.129:59855)
0808 13:04:45.590634 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:45.590652 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:45.590770 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:45.591188 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:45.591192 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:45.591629 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:45.591793 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:45.591802 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:45.592934 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:45.592938 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:45.593902 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:45.593915 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:45.594677 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:45.594681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:45.594802 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:45.595412 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:45.595428 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:45.599874 (+ 4446us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":306,"threads_started":1}
W20260808 13:04:45.600520 16415 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:45.590131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:59855 (local address 127.1.124.190:38257)
0808 13:04:45.590682 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:45.590686 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:45.590709 (+ 23us) server_negotiation.cc:407] Connection header received
0808 13:04:45.590880 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:45.590883 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:45.590946 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:45.591086 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:45.591956 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:45.592772 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:45.594081 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:45.594538 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:45.594647 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:45.600180 (+ 5533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.129:59855: BlockingRecv error: recv got EOF from 127.1.124.129:59855 (error 108)
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":317,"threads_started":1}
Aug 08 13:04:45 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:45 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:45.796445 16416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:45.786730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.131:56065)
0808 13:04:45.787337 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:45.787353 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:45.787517 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:45.787896 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:45.787901 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:45.788425 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:45.788593 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:45.788601 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:45.789623 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:45.789629 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:45.790586 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:45.790598 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:45.791344 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:45.791351 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:45.791518 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:45.792292 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:45.792319 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:45.796199 (+ 3880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":357,"threads_started":1}
W20260808 13:04:45.796591 16415 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:45.786855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:56065 (local address 127.1.124.190:38257)
0808 13:04:45.787057 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:45.787066 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:45.787440 (+ 374us) server_negotiation.cc:407] Connection header received
0808 13:04:45.787648 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:45.787653 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:45.787736 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:45.787842 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:45.788738 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:45.789474 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:45.790751 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:45.791155 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:45.791286 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:45.796383 (+ 5097us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.131:56065: BlockingRecv error: recv got EOF from 127.1.124.131:56065 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 08 13:04:45 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:45 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:45.986372 16418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:45.976046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.130:34477)
0808 13:04:45.976737 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:45.976756 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:45.976909 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:45.977287 (+ 378us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:45.977291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:45.977818 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:45.977984 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:45.977993 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:45.979215 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:45.979219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:45.980329 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:45.980342 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:45.981011 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:45.981017 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:45.981154 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:45.981849 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:45.981868 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:45.986096 (+ 4228us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":372,"threads_started":1}
W20260808 13:04:45.986542 16415 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:45.976247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:34477 (local address 127.1.124.190:38257)
0808 13:04:45.976417 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:45.976421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:45.976835 (+ 414us) server_negotiation.cc:407] Connection header received
0808 13:04:45.977023 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:45.977027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:45.977126 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:45.977230 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:45.978153 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:45.979080 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:45.980469 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:45.980880 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:45.980974 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:45.986287 (+ 5313us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.130:34477: BlockingRecv error: recv got EOF from 127.1.124.130:34477 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 13:04:46 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:46 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.129@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:46.610714 16420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:46.601715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.129:38865)
0808 13:04:46.602248 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:46.602265 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:46.602375 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:46.602746 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:46.602749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:46.603122 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:46.603328 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:46.603337 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:46.604390 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:46.604394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:46.605319 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:46.605330 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:46.606008 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:46.606012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:46.606137 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:46.606764 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:46.606781 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:46.610336 (+ 3555us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":269,"threads_started":1}
W20260808 13:04:46.610811 16421 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:46.601761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:38865 (local address 127.1.124.190:38257)
0808 13:04:46.602292 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:46.602297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:46.602314 (+ 17us) server_negotiation.cc:407] Connection header received
0808 13:04:46.602513 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:46.602517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:46.602604 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:46.602698 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:46.603454 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:46.604256 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:46.605443 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:46.605884 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:46.605980 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:46.610542 (+ 4562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.129:38865: BlockingRecv error: recv got EOF from 127.1.124.129:38865 (error 108)
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":309,"threads_started":1}
Aug 08 13:04:46 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:46 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.131@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:46.806856 16422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:46.797605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.131:40061)
0808 13:04:46.798136 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:46.798153 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:46.798269 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:46.798599 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:46.798603 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:46.799018 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:46.799174 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:46.799182 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:46.800259 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:46.800263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:46.801224 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:46.801238 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:46.801930 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:46.801934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:46.802066 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:46.802741 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:46.802759 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:46.806621 (+ 3862us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":300,"threads_started":1}
W20260808 13:04:46.807121 16421 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:46.797720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:40061 (local address 127.1.124.190:38257)
0808 13:04:46.797961 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:46.797966 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:46.798232 (+ 266us) server_negotiation.cc:407] Connection header received
0808 13:04:46.798375 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:46.798379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:46.798442 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:46.798536 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:46.799309 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:46.800120 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:46.801367 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:46.801775 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:46.801888 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:46.806828 (+ 4940us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.131:40061: BlockingRecv error: recv got EOF from 127.1.124.131:40061 (error 108)
Metrics: {"server-negotiator.queue_time_us":119}
Aug 08 13:04:46 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
Aug 08 13:04:46 dist-test-slave-t3q3 krb5kdc[14660](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.1.124.130@KRBTEST.COM for oryx/127.1.124.190@KRBTEST.COM, Server not found in Kerberos database
I20260808 13:04:46.996824 16423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:46.987730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38257 (local address 127.1.124.130:49689)
0808 13:04:46.988272 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:46.988288 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:46.988404 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:46.988753 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:46.988756 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:46.989114 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:46.989303 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:46.989311 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:46.990333 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:46.990337 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:46.991272 (+ 935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:46.991282 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:46.991998 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:46.992004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:46.992244 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:46.992853 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:46.992871 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:46.996548 (+ 3677us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.1.124.190:38257: Server oryx/127.1.124.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":271,"threads_started":1}
W20260808 13:04:46.997058 16421 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:46.987799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:49689 (local address 127.1.124.190:38257)
0808 13:04:46.987992 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:46.987997 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:46.988359 (+ 362us) server_negotiation.cc:407] Connection header received
0808 13:04:46.988508 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:46.988512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:46.988605 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:46.988698 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:46.989430 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:46.990207 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:46.991405 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:46.991835 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:46.991935 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:46.996769 (+ 4834us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.1.124.130:49689: BlockingRecv error: recv got EOF from 127.1.124.130:49689 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 13:04:47.520408 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 15634
I20260808 13:04:47.533721 16426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:47.521679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58446 (local address 127.1.124.129:33267)
0808 13:04:47.522409 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:47.522416 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:47.522443 (+ 27us) server_negotiation.cc:407] Connection header received
0808 13:04:47.522514 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:47.522520 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:47.522752 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:47.522927 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:47.524509 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:47.525611 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:47.526891 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:47.527482 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:47.527598 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:47.530259 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:47.530267 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:47.530281 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:47.530332 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:47.533218 (+ 2886us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.1.124.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 13:04:47.533505 (+ 287us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58446: Request ticket server kudu/127.1.124.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":402,"threads_started":1}
W20260808 13:04:47.533855 16426 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58446: Request ticket server kudu/127.1.124.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 13:04:47.533998 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 15634: Not authorized: Client connection negotiation failed: client connection to 127.1.124.129:33267: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.1.124.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 13:04:47.534198 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 15634
I20260808 13:04:47.565932 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 15894
I20260808 13:04:47.579561 16427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:47.567275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45390 (local address 127.1.124.130:41185)
0808 13:04:47.567904 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:47.567913 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:47.567940 (+ 27us) server_negotiation.cc:407] Connection header received
0808 13:04:47.568006 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:47.568012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:47.568354 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:47.568583 (+ 229us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:47.569994 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:47.571093 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:47.572463 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:47.573063 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:47.573212 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:47.575805 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:47.575814 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:47.575829 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:47.575875 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:47.578833 (+ 2958us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.1.124.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 13:04:47.579234 (+ 401us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45390: Request ticket server kudu/127.1.124.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":321,"threads_started":1}
W20260808 13:04:47.579599 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 15894: Not authorized: Client connection negotiation failed: client connection to 127.1.124.130:41185: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.1.124.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260808 13:04:47.579686 16427 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45390: Request ticket server kudu/127.1.124.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 13:04:47.579775 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 15894
I20260808 13:04:47.606180 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 16119
I20260808 13:04:47.620451 16428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:47.607804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41666 (local address 127.1.124.131:36891)
0808 13:04:47.608530 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:47.608542 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:47.608577 (+ 35us) server_negotiation.cc:407] Connection header received
0808 13:04:47.608664 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:47.608673 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:47.609017 (+ 344us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:47.609268 (+ 251us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:47.610603 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:47.611659 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:47.612992 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:47.613865 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:47.613970 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:47.616811 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:47.616820 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:47.616835 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:47.616890 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:47.619617 (+ 2727us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.1.124.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0808 13:04:47.619993 (+ 376us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:41666: Request ticket server kudu/127.1.124.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":329,"threads_started":1}
W20260808 13:04:47.620605 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 16119: Not authorized: Client connection negotiation failed: client connection to 127.1.124.131:36891: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.1.124.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 13:04:47.620785 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 16119
W20260808 13:04:47.620783 16428 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:41666: Request ticket server kudu/127.1.124.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260808 13:04:47.645364 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 15522
I20260808 13:04:47.646812 15625 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:47.767884 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 15522
2026-08-08T13:04:47Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22151 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T13:04:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:04:47Z Disabled control of system clock
I20260808 13:04:47.864565 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:46027
--webserver_interface=127.1.124.190
--webserver_port=0
--builtin_ntp_servers=127.1.124.148:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:46027
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 13:04:48.210737 16434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:48.211052 16434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:48.211230 16434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:48.218968 16434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:04:48.219125 16434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:48.219281 16434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:48.219388 16434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:04:48.219458 16434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:04:48.232383 16434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:46027
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:46027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:48.234157 16434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:48.236531 16434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:48.249269 16440 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:48.249289 16441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:48.249516 16443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:48.250003 16434 server_base.cc:1034] running on GCE node
I20260808 13:04:48.250854 16434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:48.252451 16434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:48.253696 16434 hybrid_clock.cc:648] HybridClock initialized: now 1786194288253640 us; error 49 us; skew 500 ppm
I20260808 13:04:48.254086 16434 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:04:48.259711 16434 webserver.cc:460] Webserver started at http://127.1.124.190:39207/ using document root <none> and password file <none>
I20260808 13:04:48.260567 16434 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:48.260710 16434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:48.261068 16434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:48.264117 16434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "56212b2901624e24879efddec15c6b6e"
format_stamp: "Formatted at 2026-08-08 13:04:48 on dist-test-slave-t3q3"
server_key: "c52add38ba11052f93d4ca2f7e0892a6"
server_key_iv: "cee5fc6e53b4b7488c8dd2565c07005c"
server_key_version: "encryptionkey@0"
I20260808 13:04:48.264933 16434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "56212b2901624e24879efddec15c6b6e"
format_stamp: "Formatted at 2026-08-08 13:04:48 on dist-test-slave-t3q3"
server_key: "c52add38ba11052f93d4ca2f7e0892a6"
server_key_iv: "cee5fc6e53b4b7488c8dd2565c07005c"
server_key_version: "encryptionkey@0"
I20260808 13:04:48.270352 16434 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:04:48.274087 16449 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:48.275516 16434 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:04:48.275736 16434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "56212b2901624e24879efddec15c6b6e"
format_stamp: "Formatted at 2026-08-08 13:04:48 on dist-test-slave-t3q3"
server_key: "c52add38ba11052f93d4ca2f7e0892a6"
server_key_iv: "cee5fc6e53b4b7488c8dd2565c07005c"
server_key_version: "encryptionkey@0"
I20260808 13:04:48.275977 16434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:48.287919 16434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:48.288764 16434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:48.289076 16434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:48.308097 16434 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:46027
I20260808 13:04:48.308176 16533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:46027 every 8 connection(s)
I20260808 13:04:48.309554 16434 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:04:48.312930 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 16434
I20260808 13:04:48.313182 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:04:48.314077 1522 external_mini_cluster.cc:1428] Setting key ef00f712903b2f05b9fee0055422b88c
I20260808 13:04:48.314098 16534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:48.329315 16534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e: Bootstrap starting.
I20260808 13:04:48.329607 16538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:48.316165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43222 (local address 127.1.124.190:46027)
0808 13:04:48.317115 (+ 950us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:48.317127 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:48.317188 (+ 61us) server_negotiation.cc:407] Connection header received
0808 13:04:48.319585 (+ 2397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:48.319609 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:48.320184 (+ 575us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:48.320543 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:48.321362 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:48.322797 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:48.324743 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:48.325423 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:48.325561 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:48.327009 (+ 1448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:48.327021 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:48.327036 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:04:48.327624 (+ 588us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:48.327709 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:04:48.327736 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:48.327837 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:48.328199 (+ 362us) server_negotiation.cc:299] Negotiation successful
0808 13:04:48.328551 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":331,"threads_started":1}
I20260808 13:04:48.333232 16534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:48.334270 16534 log.cc:824] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e: Log is configured to *not* fsync() on all Append() calls
I20260808 13:04:48.337445 16534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e: No bootstrap required, opened a new log
I20260808 13:04:48.345341 16534 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56212b2901624e24879efddec15c6b6e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 46027 } }
I20260808 13:04:48.345760 16534 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:48.345873 16534 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56212b2901624e24879efddec15c6b6e, State: Initialized, Role: FOLLOWER
I20260808 13:04:48.346531 16534 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56212b2901624e24879efddec15c6b6e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 46027 } }
I20260808 13:04:48.346827 16534 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:04:48.347002 16534 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:04:48.347211 16534 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:48.349850 16534 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56212b2901624e24879efddec15c6b6e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 46027 } }
I20260808 13:04:48.350368 16534 leader_election.cc:302] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56212b2901624e24879efddec15c6b6e; no voters:
I20260808 13:04:48.350989 16534 leader_election.cc:288] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:04:48.351162 16540 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:04:48.352089 16540 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e [term 1 LEADER]: Becoming Leader. State: Replica: 56212b2901624e24879efddec15c6b6e, State: Running, Role: LEADER
I20260808 13:04:48.352650 16540 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56212b2901624e24879efddec15c6b6e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 46027 } }
I20260808 13:04:48.353385 16534 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:04:48.355836 16543 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56212b2901624e24879efddec15c6b6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56212b2901624e24879efddec15c6b6e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 46027 } } }
I20260808 13:04:48.356248 16543 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:48.355940 16544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56212b2901624e24879efddec15c6b6e. Latest consensus state: current_term: 1 leader_uuid: "56212b2901624e24879efddec15c6b6e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56212b2901624e24879efddec15c6b6e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 46027 } } }
I20260808 13:04:48.357002 16544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:48.359086 16549 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:04:48.364704 16549 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:04:48.373708 16549 catalog_manager.cc:1347] Generated new cluster ID: 4c10779536024a2781a98a4c3b858d1a
I20260808 13:04:48.373839 16549 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:04:48.392019 16549 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:04:48.393368 16549 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:04:48.399345 16549 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56212b2901624e24879efddec15c6b6e: Generated new TSK 0
I20260808 13:04:48.400173 16549 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:04:48.415831 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:46027
--builtin_ntp_servers=127.1.124.148:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 13:04:48.771194 16568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:48.771503 16568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:48.771601 16568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:48.778924 16568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:48.779034 16568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:48.779206 16568 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:04:48.788816 16568 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:46027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:48.790115 16568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:48.791848 16568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:48.806226 16583 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:48.806236 16580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:48.806360 16578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:48.807032 16568 server_base.cc:1034] running on GCE node
I20260808 13:04:48.807631 16568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:48.808614 16568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:48.809831 16568 hybrid_clock.cc:648] HybridClock initialized: now 1786194288809792 us; error 33 us; skew 500 ppm
I20260808 13:04:48.810189 16568 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:04:48.812567 16568 webserver.cc:460] Webserver started at http://127.1.124.129:44307/ using document root <none> and password file <none>
I20260808 13:04:48.813308 16568 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:48.813436 16568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:48.813789 16568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:48.816854 16568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "e31ef1a63d2a4ee3ac1119e4f2b5da61"
format_stamp: "Formatted at 2026-08-08 13:04:48 on dist-test-slave-t3q3"
server_key: "5ecab6b2034e33f1ad207ffd050cec4e"
server_key_iv: "c9a31448cdd1d49736a0d120f6b51167"
server_key_version: "encryptionkey@0"
I20260808 13:04:48.817674 16568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "e31ef1a63d2a4ee3ac1119e4f2b5da61"
format_stamp: "Formatted at 2026-08-08 13:04:48 on dist-test-slave-t3q3"
server_key: "5ecab6b2034e33f1ad207ffd050cec4e"
server_key_iv: "c9a31448cdd1d49736a0d120f6b51167"
server_key_version: "encryptionkey@0"
I20260808 13:04:48.822985 16568 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:04:48.826752 16593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:48.828178 16568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:04:48.828359 16568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "e31ef1a63d2a4ee3ac1119e4f2b5da61"
format_stamp: "Formatted at 2026-08-08 13:04:48 on dist-test-slave-t3q3"
server_key: "5ecab6b2034e33f1ad207ffd050cec4e"
server_key_iv: "c9a31448cdd1d49736a0d120f6b51167"
server_key_version: "encryptionkey@0"
I20260808 13:04:48.828603 16568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:48.845268 16568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:48.846233 16568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:48.846562 16568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:48.847786 16568 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:48.849375 16568 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:48.849511 16568 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:48.849647 16568 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:48.849736 16568 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:48.884514 16568 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:33939
I20260808 13:04:48.884608 16762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:33939 every 8 connection(s)
I20260808 13:04:48.885852 16568 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:04:48.887151 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 16568
I20260808 13:04:48.887373 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:04:48.888243 1522 external_mini_cluster.cc:1428] Setting key 74e09c98296419db870a55d72f26c664
I20260808 13:04:48.891299 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:46027
--builtin_ntp_servers=127.1.124.148:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 13:04:48.900262 16765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:48.888881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:36059 (local address 127.1.124.190:46027)
0808 13:04:48.889414 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:48.889419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:48.890751 (+ 1332us) server_negotiation.cc:407] Connection header received
0808 13:04:48.892929 (+ 2178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:48.892935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:48.893046 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:48.893176 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:48.894271 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:48.895023 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:48.896729 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:48.897466 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:48.897624 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:48.899115 (+ 1491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:48.899124 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:48.899131 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:04:48.899159 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:48.899211 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:04:48.899230 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:48.899333 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:48.899661 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 13:04:48.899779 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":291,"threads_started":1}
I20260808 13:04:48.901197 16766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:48.889517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:46027 (local address 127.1.124.129:36059)
0808 13:04:48.890599 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:48.890637 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:48.892569 (+ 1932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:48.893430 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:48.893441 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:48.893539 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:48.894099 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:48.894117 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:48.895200 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:48.895206 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:48.896568 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:48.896586 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:48.897699 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:48.897707 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:48.898016 (+ 309us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:48.898788 (+ 772us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:04:48.898791 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:48.898838 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:04:48.898918 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:48.899362 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:48.899372 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:48.899379 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 13:04:48.899636 (+ 257us) client_negotiation.cc:240] Negotiation successful
0808 13:04:48.899971 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":305,"threads_started":1}
I20260808 13:04:48.903539 16763 heartbeater.cc:347] Connected to a master server at 127.1.124.190:46027
I20260808 13:04:48.903855 16763 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:48.904642 16763 heartbeater.cc:511] Master 127.1.124.190:46027 requested a full tablet report, sending...
I20260808 13:04:48.906641 16480 ts_manager.cc:194] Registered new tserver with Master: e31ef1a63d2a4ee3ac1119e4f2b5da61 (127.1.124.129:33939)
I20260808 13:04:48.908205 16480 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.124.129:36059
W20260808 13:04:49.239673 16768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:49.240015 16768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:49.240201 16768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:49.247480 16768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:49.247632 16768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:49.247812 16768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:04:49.257093 16768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:46027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:49.258529 16768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:49.260510 16768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:49.274005 16776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:49.274037 16778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:49.274658 16768 server_base.cc:1034] running on GCE node
W20260808 13:04:49.274768 16780 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:49.275529 16768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:49.276585 16768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:49.277792 16768 hybrid_clock.cc:648] HybridClock initialized: now 1786194289277737 us; error 44 us; skew 500 ppm
I20260808 13:04:49.278088 16768 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:04:49.280524 16768 webserver.cc:460] Webserver started at http://127.1.124.130:38467/ using document root <none> and password file <none>
I20260808 13:04:49.281217 16768 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:49.281308 16768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:49.281595 16768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:49.284746 16768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "20840e5e2bf84f61a8b08d3cd9793967"
format_stamp: "Formatted at 2026-08-08 13:04:49 on dist-test-slave-t3q3"
server_key: "48e32865719753677f7ce8ac5a8c1c61"
server_key_iv: "0095c52626e7647697e6d7a1f98d6eb1"
server_key_version: "encryptionkey@0"
I20260808 13:04:49.285508 16768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "20840e5e2bf84f61a8b08d3cd9793967"
format_stamp: "Formatted at 2026-08-08 13:04:49 on dist-test-slave-t3q3"
server_key: "48e32865719753677f7ce8ac5a8c1c61"
server_key_iv: "0095c52626e7647697e6d7a1f98d6eb1"
server_key_version: "encryptionkey@0"
I20260808 13:04:49.290822 16768 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:04:49.294567 16790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:49.296444 16768 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:04:49.296624 16768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "20840e5e2bf84f61a8b08d3cd9793967"
format_stamp: "Formatted at 2026-08-08 13:04:49 on dist-test-slave-t3q3"
server_key: "48e32865719753677f7ce8ac5a8c1c61"
server_key_iv: "0095c52626e7647697e6d7a1f98d6eb1"
server_key_version: "encryptionkey@0"
I20260808 13:04:49.296845 16768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:49.342237 16768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:49.343124 16768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:49.343472 16768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:49.344561 16768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:49.346048 16768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:49.346174 16768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:49.346309 16768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:49.346410 16768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:49.379981 16768 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:36433
I20260808 13:04:49.380029 16970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:36433 every 8 connection(s)
I20260808 13:04:49.381408 16768 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:04:49.384466 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 16768
I20260808 13:04:49.384646 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:04:49.385214 1522 external_mini_cluster.cc:1428] Setting key 62c9024f5bbd794d5556c28670a6364b
I20260808 13:04:49.388406 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:46027
--builtin_ntp_servers=127.1.124.148:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 13:04:49.395493 16765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:49.383964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:37931 (local address 127.1.124.190:46027)
0808 13:04:49.384234 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:49.384243 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:49.385950 (+ 1707us) server_negotiation.cc:407] Connection header received
0808 13:04:49.387819 (+ 1869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:49.387823 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:49.387888 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:49.387968 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:49.389433 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:49.390167 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:49.392609 (+ 2442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:49.392992 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:49.393109 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:49.394717 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:49.394721 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:49.394725 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:04:49.394743 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:49.394783 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:04:49.394795 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:49.394866 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:49.395190 (+ 324us) server_negotiation.cc:299] Negotiation successful
0808 13:04:49.395270 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260808 13:04:49.397011 16979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:49.384650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:46027 (local address 127.1.124.130:37931)
0808 13:04:49.385785 (+ 1135us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:49.385829 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:49.387520 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:49.388270 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:49.388281 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:49.388403 (+ 122us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:49.389241 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:49.389270 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:49.390330 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:49.390338 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:49.392475 (+ 2137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:49.392492 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:49.393218 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:49.393228 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:49.393547 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:49.394338 (+ 791us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:04:49.394344 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:49.394409 (+ 65us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:04:49.394509 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:49.394893 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:49.394906 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:49.394916 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 13:04:49.395135 (+ 219us) client_negotiation.cc:240] Negotiation successful
0808 13:04:49.395554 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":312,"threads_started":1}
I20260808 13:04:49.398447 16974 heartbeater.cc:347] Connected to a master server at 127.1.124.190:46027
I20260808 13:04:49.398769 16974 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:49.399567 16974 heartbeater.cc:511] Master 127.1.124.190:46027 requested a full tablet report, sending...
I20260808 13:04:49.401196 16480 ts_manager.cc:194] Registered new tserver with Master: 20840e5e2bf84f61a8b08d3cd9793967 (127.1.124.130:36433)
I20260808 13:04:49.402155 16480 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.124.130:37931
W20260808 13:04:49.734457 16980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:49.734815 16980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:49.734979 16980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:49.742663 16980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:49.742812 16980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:49.742995 16980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:04:49.752379 16980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:46027
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:49.753829 16980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:49.755671 16980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:49.769202 16987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:49.769167 16990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:49.769591 16995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:49.770040 16980 server_base.cc:1034] running on GCE node
I20260808 13:04:49.770673 16980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:49.771660 16980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:49.772861 16980 hybrid_clock.cc:648] HybridClock initialized: now 1786194289772816 us; error 31 us; skew 500 ppm
I20260808 13:04:49.773211 16980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:04:49.775583 16980 webserver.cc:460] Webserver started at http://127.1.124.131:46865/ using document root <none> and password file <none>
I20260808 13:04:49.776367 16980 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:49.776489 16980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:49.776852 16980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:49.779976 16980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "9eaa17fe68b041e3b25071b62fdd9ed6"
format_stamp: "Formatted at 2026-08-08 13:04:49 on dist-test-slave-t3q3"
server_key: "2fc1f5311bda44a5414b84a8cc97540a"
server_key_iv: "a730356d9cf9b1a514606ea84398568b"
server_key_version: "encryptionkey@0"
I20260808 13:04:49.780963 16980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "9eaa17fe68b041e3b25071b62fdd9ed6"
format_stamp: "Formatted at 2026-08-08 13:04:49 on dist-test-slave-t3q3"
server_key: "2fc1f5311bda44a5414b84a8cc97540a"
server_key_iv: "a730356d9cf9b1a514606ea84398568b"
server_key_version: "encryptionkey@0"
I20260808 13:04:49.786303 16980 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:04:49.790100 17002 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:49.791566 16980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:04:49.791780 16980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "9eaa17fe68b041e3b25071b62fdd9ed6"
format_stamp: "Formatted at 2026-08-08 13:04:49 on dist-test-slave-t3q3"
server_key: "2fc1f5311bda44a5414b84a8cc97540a"
server_key_iv: "a730356d9cf9b1a514606ea84398568b"
server_key_version: "encryptionkey@0"
I20260808 13:04:49.792018 16980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:49.809834 16980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:49.810736 16980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:49.811065 16980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:49.812125 16980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:49.813669 16980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:49.813758 16980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:49.813923 16980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:49.814020 16980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:49.848662 16980 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:42311
I20260808 13:04:49.848718 17215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:42311 every 8 connection(s)
I20260808 13:04:49.850066 16980 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:04:49.850705 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 16980
I20260808 13:04:49.850916 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:04:49.851548 1522 external_mini_cluster.cc:1428] Setting key 05ebdf1b31f06e8f6b61ae82e6bd7e20
I20260808 13:04:49.863412 16765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:49.853026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:48693 (local address 127.1.124.190:46027)
0808 13:04:49.853231 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:49.853235 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:49.855163 (+ 1928us) server_negotiation.cc:407] Connection header received
0808 13:04:49.857001 (+ 1838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:49.857006 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:49.857091 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:49.857209 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:49.858257 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:49.859035 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:49.860735 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:49.861144 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:49.861276 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:49.862599 (+ 1323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:49.862604 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:49.862607 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:04:49.862624 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:49.862654 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:04:49.862672 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:49.862762 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:49.862985 (+ 223us) server_negotiation.cc:299] Negotiation successful
0808 13:04:49.863080 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 13:04:49.864477 17218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:49.853847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:46027 (local address 127.1.124.131:48693)
0808 13:04:49.854998 (+ 1151us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:49.855044 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:49.856744 (+ 1700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:49.857460 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:49.857468 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:49.857552 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:49.858088 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:49.858104 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:49.859202 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:49.859206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:49.860584 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:49.860606 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:49.861362 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:49.861370 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:49.861625 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:49.862281 (+ 656us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:04:49.862284 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:49.862321 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:04:49.862393 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:49.862764 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:49.862778 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:49.862788 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 13:04:49.862972 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 13:04:49.863353 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":365,"threads_started":1}
I20260808 13:04:49.866001 17216 heartbeater.cc:347] Connected to a master server at 127.1.124.190:46027
I20260808 13:04:49.866324 17216 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:49.867111 17216 heartbeater.cc:511] Master 127.1.124.190:46027 requested a full tablet report, sending...
I20260808 13:04:49.868548 16480 ts_manager.cc:194] Registered new tserver with Master: 9eaa17fe68b041e3b25071b62fdd9ed6 (127.1.124.131:42311)
I20260808 13:04:49.869345 16480 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.124.131:48693
I20260808 13:04:49.879438 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 13:04:49.886075 16765 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:49.884365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55206 (local address 127.1.124.190:46027)
0808 13:04:49.884584 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:49.884590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:49.885119 (+ 529us) server_negotiation.cc:407] Connection header received
0808 13:04:49.885242 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:49.885246 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:49.885338 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:49.885448 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:49.885823 (+ 375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55206: BlockingRecv error: recv got EOF from 127.0.0.1:55206 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 13:04:49.886862 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 16568
I20260808 13:04:49.895438 17228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:49.888019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36678 (local address 127.1.124.129:33939)
0808 13:04:49.888646 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:49.888654 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:49.889004 (+ 350us) server_negotiation.cc:407] Connection header received
0808 13:04:49.889090 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:49.889098 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:49.889364 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:49.889531 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:49.889844 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:49.891185 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:49.892579 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:49.893108 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:49.893214 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:49.894205 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:49.894212 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:49.894224 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:04:49.894732 (+ 508us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:49.894776 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:04:49.894799 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:49.894886 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:49.895031 (+ 145us) server_negotiation.cc:299] Negotiation successful
0808 13:04:49.895144 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":463,"thread_start_us":330,"threads_started":1}
I20260808 13:04:49.896711 16758 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:49.978369 16763 heartbeater.cc:503] Master 127.1.124.190:46027 was elected leader, sending a full tablet report...
I20260808 13:04:49.979799 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 16568
W20260808 13:04:50.005347 16456 connection.cc:535] server connection from 127.1.124.129:36059 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 13:04:50.005769 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 16768
I20260808 13:04:50.014274 17230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:50.006822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40462 (local address 127.1.124.130:36433)
0808 13:04:50.007374 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:50.007381 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:50.007408 (+ 27us) server_negotiation.cc:407] Connection header received
0808 13:04:50.007467 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:50.007473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:50.007703 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:50.007934 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:50.008370 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:50.009391 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:50.010684 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:50.011257 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:50.011403 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:50.012681 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:50.012690 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:50.012705 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:04:50.013359 (+ 654us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:50.013410 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:04:50.013432 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:50.013603 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:50.013759 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 13:04:50.013902 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":297,"threads_started":1}
I20260808 13:04:50.015681 16962 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:50.100504 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 16768
I20260808 13:04:50.126453 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 16980
I20260808 13:04:50.135113 17232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:50.127638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34418 (local address 127.1.124.131:42311)
0808 13:04:50.128463 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:50.128472 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:50.128506 (+ 34us) server_negotiation.cc:407] Connection header received
0808 13:04:50.128577 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:50.128584 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:50.128809 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:50.128958 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:50.129361 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:50.130547 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:50.131812 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:50.132481 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:50.132587 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:50.133630 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:50.133638 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:50.133652 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:04:50.134270 (+ 618us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:50.134323 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:04:50.134343 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:50.134433 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:50.134616 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 13:04:50.134760 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":480,"threads_started":1}
I20260808 13:04:50.136524 17209 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:50.219020 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 16980
I20260808 13:04:50.244561 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 16434
I20260808 13:04:50.246089 16529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:50.351647 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 16434
2026-08-08T13:04:50Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2577 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T13:04:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:04:50Z Disabled control of system clock
I20260808 13:04:50.444558 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:45151
--webserver_interface=127.1.124.190
--webserver_port=0
--builtin_ntp_servers=127.1.124.148:39043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:45151
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 13:04:50.791258 17241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:50.791549 17241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:50.791636 17241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:50.798916 17241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:04:50.799047 17241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:50.799088 17241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:50.799147 17241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:04:50.799193 17241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:04:50.808568 17241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:39043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:45151
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:45151
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:50.809863 17241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:50.811636 17241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:50.823750 17248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:50.823709 17254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:50.823812 17250 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:50.824740 17241 server_base.cc:1034] running on GCE node
I20260808 13:04:50.825562 17241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:50.827009 17241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:50.828250 17241 hybrid_clock.cc:648] HybridClock initialized: now 1786194290828203 us; error 36 us; skew 500 ppm
I20260808 13:04:50.828615 17241 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:04:50.831120 17241 webserver.cc:460] Webserver started at http://127.1.124.190:45335/ using document root <none> and password file <none>
I20260808 13:04:50.831881 17241 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:50.832024 17241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:50.832410 17241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:50.837826 17241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "a3b547f409cc4d2fb1d2d5754ad2a45e"
format_stamp: "Formatted at 2026-08-08 13:04:50 on dist-test-slave-t3q3"
server_key: "7400023ad6e9caadb2098f46a902d517"
server_key_iv: "1715eded8a351a58f4ff7e957e2b2e15"
server_key_version: "encryptionkey@0"
I20260808 13:04:50.838765 17241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "a3b547f409cc4d2fb1d2d5754ad2a45e"
format_stamp: "Formatted at 2026-08-08 13:04:50 on dist-test-slave-t3q3"
server_key: "7400023ad6e9caadb2098f46a902d517"
server_key_iv: "1715eded8a351a58f4ff7e957e2b2e15"
server_key_version: "encryptionkey@0"
I20260808 13:04:50.844558 17241 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 13:04:50.848186 17266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:50.849525 17241 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:04:50.849718 17241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "a3b547f409cc4d2fb1d2d5754ad2a45e"
format_stamp: "Formatted at 2026-08-08 13:04:50 on dist-test-slave-t3q3"
server_key: "7400023ad6e9caadb2098f46a902d517"
server_key_iv: "1715eded8a351a58f4ff7e957e2b2e15"
server_key_version: "encryptionkey@0"
I20260808 13:04:50.849947 17241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:50.855885 17241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:50.856778 17241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:50.857095 17241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:50.876355 17241 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:45151
I20260808 13:04:50.876394 17358 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:45151 every 8 connection(s)
I20260808 13:04:50.877764 17241 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:04:50.881608 17359 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:50.882761 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 17241
I20260808 13:04:50.882992 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:04:50.883661 1522 external_mini_cluster.cc:1428] Setting key 5e2a2810fcc3e0879823a56c8328ff3d
I20260808 13:04:50.892122 17366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:50.885766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43284 (local address 127.1.124.190:45151)
0808 13:04:50.886840 (+ 1074us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:50.886851 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:50.886894 (+ 43us) server_negotiation.cc:407] Connection header received
0808 13:04:50.888511 (+ 1617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:50.888534 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:50.888952 (+ 418us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:50.889303 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:50.890462 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:50.890468 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:50.890480 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:04:50.890495 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:50.890555 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:04:50.890570 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:50.890745 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:50.890896 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 13:04:50.891193 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":411,"threads_started":1}
I20260808 13:04:50.895874 17359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e: Bootstrap starting.
I20260808 13:04:50.899077 17359 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:50.900146 17359 log.cc:824] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e: Log is configured to *not* fsync() on all Append() calls
I20260808 13:04:50.902807 17359 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e: No bootstrap required, opened a new log
I20260808 13:04:50.910254 17359 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3b547f409cc4d2fb1d2d5754ad2a45e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 45151 } }
I20260808 13:04:50.910625 17359 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:50.910735 17359 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3b547f409cc4d2fb1d2d5754ad2a45e, State: Initialized, Role: FOLLOWER
I20260808 13:04:50.911360 17359 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3b547f409cc4d2fb1d2d5754ad2a45e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 45151 } }
I20260808 13:04:50.911602 17359 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:04:50.911756 17359 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:04:50.911939 17359 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:50.914120 17359 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3b547f409cc4d2fb1d2d5754ad2a45e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 45151 } }
I20260808 13:04:50.914618 17359 leader_election.cc:302] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3b547f409cc4d2fb1d2d5754ad2a45e; no voters:
I20260808 13:04:50.915329 17359 leader_election.cc:288] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:04:50.915457 17368 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:04:50.916534 17368 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e [term 1 LEADER]: Becoming Leader. State: Replica: a3b547f409cc4d2fb1d2d5754ad2a45e, State: Running, Role: LEADER
I20260808 13:04:50.917142 17368 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3b547f409cc4d2fb1d2d5754ad2a45e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 45151 } }
I20260808 13:04:50.917637 17359 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:04:50.919952 17369 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3b547f409cc4d2fb1d2d5754ad2a45e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3b547f409cc4d2fb1d2d5754ad2a45e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 45151 } } }
I20260808 13:04:50.919951 17374 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3b547f409cc4d2fb1d2d5754ad2a45e. Latest consensus state: current_term: 1 leader_uuid: "a3b547f409cc4d2fb1d2d5754ad2a45e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3b547f409cc4d2fb1d2d5754ad2a45e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 45151 } } }
I20260808 13:04:50.920401 17369 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:50.920476 17374 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:50.921631 17381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:04:50.927042 17381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:04:50.935871 17381 catalog_manager.cc:1347] Generated new cluster ID: 602a5756a0c5453399312dd3661b39cf
I20260808 13:04:50.935987 17381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:04:50.952877 17381 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:04:50.953297 17381 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:04:50.961436 17381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3b547f409cc4d2fb1d2d5754ad2a45e: Generated new TSK 0
I20260808 13:04:50.962383 17381 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:04:50.974740 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:45151
--builtin_ntp_servers=127.1.124.148:39043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 13:04:51.321434 17399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:51.321709 17399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:51.321882 17399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:51.329279 17399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:51.329380 17399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:51.329540 17399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:04:51.338574 17399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:39043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:45151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:51.340023 17399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:51.341847 17399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:51.355962 17411 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:51.355913 17412 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:51.355962 17414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:51.356637 17399 server_base.cc:1034] running on GCE node
I20260808 13:04:51.357285 17399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:51.358250 17399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:51.359486 17399 hybrid_clock.cc:648] HybridClock initialized: now 1786194291359446 us; error 54 us; skew 500 ppm
I20260808 13:04:51.359860 17399 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:04:51.362172 17399 webserver.cc:460] Webserver started at http://127.1.124.129:43265/ using document root <none> and password file <none>
I20260808 13:04:51.362875 17399 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:51.363008 17399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:51.363354 17399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:51.366206 17399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "80b76737065845d3b55179a356d96332"
format_stamp: "Formatted at 2026-08-08 13:04:51 on dist-test-slave-t3q3"
server_key: "dd799db86f5c09374e7d865d160b87e8"
server_key_iv: "d3b67b604e2ba72535e99ebd1d52adac"
server_key_version: "encryptionkey@0"
I20260808 13:04:51.366986 17399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "80b76737065845d3b55179a356d96332"
format_stamp: "Formatted at 2026-08-08 13:04:51 on dist-test-slave-t3q3"
server_key: "dd799db86f5c09374e7d865d160b87e8"
server_key_iv: "d3b67b604e2ba72535e99ebd1d52adac"
server_key_version: "encryptionkey@0"
I20260808 13:04:51.371847 17399 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260808 13:04:51.375409 17421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:51.376796 17399 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:04:51.376979 17399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "80b76737065845d3b55179a356d96332"
format_stamp: "Formatted at 2026-08-08 13:04:51 on dist-test-slave-t3q3"
server_key: "dd799db86f5c09374e7d865d160b87e8"
server_key_iv: "d3b67b604e2ba72535e99ebd1d52adac"
server_key_version: "encryptionkey@0"
I20260808 13:04:51.377209 17399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:51.382828 17399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:51.383600 17399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:51.383880 17399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:51.384856 17399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:51.386229 17399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:51.386353 17399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:51.386502 17399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:51.386598 17399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:51.421800 17399 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:36105
I20260808 13:04:51.421897 17608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:36105 every 8 connection(s)
I20260808 13:04:51.423360 17399 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:04:51.431501 17616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:51.425993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:60209 (local address 127.1.124.190:45151)
0808 13:04:51.426468 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:51.426474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:51.427683 (+ 1209us) server_negotiation.cc:407] Connection header received
0808 13:04:51.429469 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:51.429473 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:51.429523 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:51.429619 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:51.430873 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:51.430877 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:51.430880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:04:51.430893 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:51.430928 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:04:51.430938 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:51.431032 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:51.431237 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 13:04:51.431290 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":232,"threads_started":1}
I20260808 13:04:51.432421 17617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:51.426619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:45151 (local address 127.1.124.129:60209)
0808 13:04:51.427525 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:51.427561 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:51.429234 (+ 1673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:51.429813 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:51.429820 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:51.429902 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:51.430604 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:04:51.430607 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:51.430643 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:04:51.430730 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:51.431019 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:51.431027 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:51.431032 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 13:04:51.431253 (+ 221us) client_negotiation.cc:240] Negotiation successful
0808 13:04:51.431432 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":271,"threads_started":1}
I20260808 13:04:51.433665 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 17399
I20260808 13:04:51.433874 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:04:51.433969 17611 heartbeater.cc:347] Connected to a master server at 127.1.124.190:45151
I20260808 13:04:51.434263 17611 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:51.434597 1522 external_mini_cluster.cc:1428] Setting key f753b7924576231d6457ac773c21adc2
I20260808 13:04:51.434872 17611 heartbeater.cc:511] Master 127.1.124.190:45151 requested a full tablet report, sending...
I20260808 13:04:51.436987 17296 ts_manager.cc:194] Registered new tserver with Master: 80b76737065845d3b55179a356d96332 (127.1.124.129:36105)
I20260808 13:04:51.437429 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:45151
--builtin_ntp_servers=127.1.124.148:39043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 13:04:51.782591 17618 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:51.782943 17618 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:51.783115 17618 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:51.790557 17618 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:51.790714 17618 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:51.790899 17618 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:04:51.800567 17618 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:39043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:45151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:51.801990 17618 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:51.803851 17618 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:51.817433 17625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:51.817466 17626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:51.817497 17630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:51.818367 17618 server_base.cc:1034] running on GCE node
I20260808 13:04:51.819043 17618 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:51.820117 17618 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:51.821357 17618 hybrid_clock.cc:648] HybridClock initialized: now 1786194291821297 us; error 53 us; skew 500 ppm
I20260808 13:04:51.821642 17618 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:04:51.824307 17618 webserver.cc:460] Webserver started at http://127.1.124.130:34021/ using document root <none> and password file <none>
I20260808 13:04:51.824975 17618 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:51.825078 17618 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:51.825358 17618 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:51.828188 17618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "c1fb9417a8db47aa9399a9872987c0e1"
format_stamp: "Formatted at 2026-08-08 13:04:51 on dist-test-slave-t3q3"
server_key: "0e1f4891e4d9b5dd708fcc4155f63224"
server_key_iv: "97f3a9248a6ce8ee3ef6b64963a2a1a6"
server_key_version: "encryptionkey@0"
I20260808 13:04:51.828920 17618 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "c1fb9417a8db47aa9399a9872987c0e1"
format_stamp: "Formatted at 2026-08-08 13:04:51 on dist-test-slave-t3q3"
server_key: "0e1f4891e4d9b5dd708fcc4155f63224"
server_key_iv: "97f3a9248a6ce8ee3ef6b64963a2a1a6"
server_key_version: "encryptionkey@0"
I20260808 13:04:51.833889 17618 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:04:51.837469 17641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:51.838860 17618 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:04:51.839044 17618 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "c1fb9417a8db47aa9399a9872987c0e1"
format_stamp: "Formatted at 2026-08-08 13:04:51 on dist-test-slave-t3q3"
server_key: "0e1f4891e4d9b5dd708fcc4155f63224"
server_key_iv: "97f3a9248a6ce8ee3ef6b64963a2a1a6"
server_key_version: "encryptionkey@0"
I20260808 13:04:51.839277 17618 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:51.845136 17618 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:51.845916 17618 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:51.846197 17618 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:51.847179 17618 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:51.848627 17618 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:51.848747 17618 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:51.848857 17618 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:51.848975 17618 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:51.883345 17618 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:43403
I20260808 13:04:51.883363 17813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:43403 every 8 connection(s)
I20260808 13:04:51.885022 17618 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:04:51.886350 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 17618
I20260808 13:04:51.886519 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:04:51.887049 1522 external_mini_cluster.cc:1428] Setting key 243562bbcef39ff75aa5e66b7fdc180e
I20260808 13:04:51.890678 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:45151
--builtin_ntp_servers=127.1.124.148:39043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 13:04:51.895852 17616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:51.887969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:59741 (local address 127.1.124.190:45151)
0808 13:04:51.888229 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:51.888235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:51.890071 (+ 1836us) server_negotiation.cc:407] Connection header received
0808 13:04:51.892874 (+ 2803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:51.892883 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:51.892950 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:51.893044 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:51.894890 (+ 1846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:51.894899 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:51.894906 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:04:51.894929 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:51.894969 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:04:51.894987 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:51.895091 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:51.895420 (+ 329us) server_negotiation.cc:299] Negotiation successful
0808 13:04:51.895491 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260808 13:04:51.897739 17818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:51.888749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:45151 (local address 127.1.124.130:59741)
0808 13:04:51.889893 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:51.889951 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:51.892546 (+ 2595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:51.893591 (+ 1045us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:51.893606 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:51.893731 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:51.894521 (+ 790us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:04:51.894527 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:51.894579 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:04:51.894694 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:51.895186 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:51.895197 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:51.895207 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 13:04:51.896250 (+ 1043us) client_negotiation.cc:240] Negotiation successful
0808 13:04:51.896508 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":338,"threads_started":1}
I20260808 13:04:51.899313 17815 heartbeater.cc:347] Connected to a master server at 127.1.124.190:45151
I20260808 13:04:51.899679 17815 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:51.900375 17815 heartbeater.cc:511] Master 127.1.124.190:45151 requested a full tablet report, sending...
I20260808 13:04:51.902021 17296 ts_manager.cc:194] Registered new tserver with Master: c1fb9417a8db47aa9399a9872987c0e1 (127.1.124.130:43403)
W20260808 13:04:52.246461 17819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:52.246758 17819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:52.246851 17819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:52.254227 17819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:52.254361 17819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:52.254504 17819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:04:52.263537 17819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:39043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:45151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:52.264792 17819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:52.266582 17819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:52.279842 17831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:52.279884 17835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:52.279911 17830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:52.280998 17819 server_base.cc:1034] running on GCE node
I20260808 13:04:52.281661 17819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:52.282665 17819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:52.283942 17819 hybrid_clock.cc:648] HybridClock initialized: now 1786194292283886 us; error 63 us; skew 500 ppm
I20260808 13:04:52.284337 17819 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:04:52.286659 17819 webserver.cc:460] Webserver started at http://127.1.124.131:46651/ using document root <none> and password file <none>
I20260808 13:04:52.287370 17819 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:52.287513 17819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:52.287864 17819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:52.290827 17819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "5c3eedac87e745eaac283375243bbc42"
format_stamp: "Formatted at 2026-08-08 13:04:52 on dist-test-slave-t3q3"
server_key: "48f50ea9d66171b5bccae84c03ca496c"
server_key_iv: "d7558e0ee04e0cc8b3b6baed42f5b65e"
server_key_version: "encryptionkey@0"
I20260808 13:04:52.291656 17819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "5c3eedac87e745eaac283375243bbc42"
format_stamp: "Formatted at 2026-08-08 13:04:52 on dist-test-slave-t3q3"
server_key: "48f50ea9d66171b5bccae84c03ca496c"
server_key_iv: "d7558e0ee04e0cc8b3b6baed42f5b65e"
server_key_version: "encryptionkey@0"
I20260808 13:04:52.297166 17819 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:04:52.300634 17841 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:52.302068 17819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:04:52.302266 17819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "5c3eedac87e745eaac283375243bbc42"
format_stamp: "Formatted at 2026-08-08 13:04:52 on dist-test-slave-t3q3"
server_key: "48f50ea9d66171b5bccae84c03ca496c"
server_key_iv: "d7558e0ee04e0cc8b3b6baed42f5b65e"
server_key_version: "encryptionkey@0"
I20260808 13:04:52.302518 17819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:52.308357 17819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:52.309159 17819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:52.309434 17819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:52.310370 17819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:52.311884 17819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:52.312016 17819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:52.312252 17819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:52.312346 17819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:52.346750 17819 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:35633
I20260808 13:04:52.346845 18028 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:35633 every 8 connection(s)
I20260808 13:04:52.348207 17819 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:04:52.353214 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 17819
I20260808 13:04:52.353399 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:04:52.353880 1522 external_mini_cluster.cc:1428] Setting key 62df2483fc4b5b9f96e0c26629e06346
I20260808 13:04:52.357366 17616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:52.351012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:44751 (local address 127.1.124.190:45151)
0808 13:04:52.351319 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:52.351326 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:52.352799 (+ 1473us) server_negotiation.cc:407] Connection header received
0808 13:04:52.354922 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:52.354928 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:52.355044 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:52.355173 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:52.356579 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:52.356585 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:52.356590 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:04:52.356613 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:52.356663 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:04:52.356679 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:52.356812 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:52.357090 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 13:04:52.357144 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260808 13:04:52.358348 18032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:52.351568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:45151 (local address 127.1.124.131:44751)
0808 13:04:52.352600 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:52.352646 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:52.354621 (+ 1975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:52.355408 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:52.355416 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:52.355510 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:52.356263 (+ 753us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:04:52.356265 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:52.356305 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:04:52.356393 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:52.356805 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:52.356817 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:52.356826 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 13:04:52.357131 (+ 305us) client_negotiation.cc:240] Negotiation successful
0808 13:04:52.357410 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"thread_start_us":312,"threads_started":1}
I20260808 13:04:52.359618 18030 heartbeater.cc:347] Connected to a master server at 127.1.124.190:45151
I20260808 13:04:52.360020 18030 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:52.360723 18030 heartbeater.cc:511] Master 127.1.124.190:45151 requested a full tablet report, sending...
I20260808 13:04:52.362161 17296 ts_manager.cc:194] Registered new tserver with Master: 5c3eedac87e745eaac283375243bbc42 (127.1.124.131:35633)
I20260808 13:04:52.369567 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 13:04:52.375793 17616 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:04:52.374208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43298 (local address 127.1.124.190:45151)
0808 13:04:52.374375 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:52.374379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:52.374865 (+ 486us) server_negotiation.cc:407] Connection header received
0808 13:04:52.375080 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:52.375084 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:52.375140 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:52.375240 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:52.375583 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43298: BlockingRecv error: recv got EOF from 127.0.0.1:43298 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 13:04:52.376855 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 17399
I20260808 13:04:52.380645 18044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:52.377963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40536 (local address 127.1.124.129:36105)
0808 13:04:52.378530 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:52.378537 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:52.378564 (+ 27us) server_negotiation.cc:407] Connection header received
0808 13:04:52.378623 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:52.378630 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:52.378859 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:52.378986 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:52.379872 (+ 886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:52.379878 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:52.379889 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:04:52.379907 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:52.379945 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:04:52.379958 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:52.380126 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:52.380264 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 13:04:52.380348 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":422,"thread_start_us":299,"threads_started":1}
I20260808 13:04:52.382009 17602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:52.464342 17611 heartbeater.cc:503] Master 127.1.124.190:45151 was elected leader, sending a full tablet report...
I20260808 13:04:52.465873 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 17399
W20260808 13:04:52.492815 17271 connection.cc:535] server connection from 127.1.124.129:60209 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 13:04:52.493135 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 17618
I20260808 13:04:52.497087 18047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:52.494174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51982 (local address 127.1.124.130:43403)
0808 13:04:52.494750 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:52.494757 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:52.494784 (+ 27us) server_negotiation.cc:407] Connection header received
0808 13:04:52.494858 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:52.494864 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:52.495080 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:52.495206 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:52.496281 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:52.496289 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:52.496301 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:04:52.496327 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:52.496376 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:04:52.496391 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:52.496513 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:52.496656 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 13:04:52.496797 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":292,"threads_started":1}
I20260808 13:04:52.498325 17807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:52.579664 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 17618
I20260808 13:04:52.604785 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 17819
I20260808 13:04:52.609175 18050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:52.605845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52632 (local address 127.1.124.131:35633)
0808 13:04:52.606520 (+ 675us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:52.606527 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:52.606570 (+ 43us) server_negotiation.cc:407] Connection header received
0808 13:04:52.606638 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:52.606645 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:52.606944 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:52.607147 (+ 203us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:52.608391 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:52.608397 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:52.608409 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:04:52.608427 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:52.608466 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:04:52.608480 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:52.608682 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:52.608762 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 13:04:52.608863 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":315,"threads_started":1}
I20260808 13:04:52.610347 18020 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:52.694770 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 17819
I20260808 13:04:52.720752 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 17241
I20260808 13:04:52.722188 17353 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:52.827571 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 17241
2026-08-08T13:04:52Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2478 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:04:52 dist-test-slave-t3q3 krb5kdc[18056](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:04:52 dist-test-slave-t3q3 krb5kdc[18056](info): set up 2 sockets
Aug 08 13:04:52 dist-test-slave-t3q3 krb5kdc[18056](info): commencing operation
krb5kdc: starting...
W20260808 13:04:54.213598 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.334s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:04:54 dist-test-slave-t3q3 krb5kdc[18056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:04:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:04:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:04:54.511868 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:46009
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:41875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:46009
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:54.862944 18079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:54.863265 18079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:54.863435 18079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:54.870812 18079 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:04:54.870925 18079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:54.871057 18079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:54.871170 18079 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:04:54.871254 18079 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:04:54.881664 18079 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:41875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:46009
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:46009
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:54.883134 18079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:54.884608 18079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:54.898048 18087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:54.897989 18094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:54.898021 18089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:54.898882 18079 server_base.cc:1034] running on GCE node
I20260808 13:04:54.900127 18079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:54.901633 18079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:54.902878 18079 hybrid_clock.cc:648] HybridClock initialized: now 1786194294902810 us; error 59 us; skew 500 ppm
I20260808 13:04:54.903252 18079 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:54 dist-test-slave-t3q3 krb5kdc[18056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194294, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:54.910877 18079 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:04:54.912544 18079 webserver.cc:460] Webserver started at http://127.1.124.190:37389/ using document root <none> and password file <none>
I20260808 13:04:54.913350 18079 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:54.913492 18079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:54.913841 18079 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:54.917084 18079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "ac63d5b391d44886a7440c696faba1fb"
format_stamp: "Formatted at 2026-08-08 13:04:54 on dist-test-slave-t3q3"
server_key: "8a95676779daffd42febcb49cd6da221"
server_key_iv: "2d6c115c7677fd37a622d12160f6c5a0"
server_key_version: "encryptionkey@0"
I20260808 13:04:54.917917 18079 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "ac63d5b391d44886a7440c696faba1fb"
format_stamp: "Formatted at 2026-08-08 13:04:54 on dist-test-slave-t3q3"
server_key: "8a95676779daffd42febcb49cd6da221"
server_key_iv: "2d6c115c7677fd37a622d12160f6c5a0"
server_key_version: "encryptionkey@0"
I20260808 13:04:54.923409 18079 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:04:54.927006 18106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:54.928434 18079 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:04:54.928633 18079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "ac63d5b391d44886a7440c696faba1fb"
format_stamp: "Formatted at 2026-08-08 13:04:54 on dist-test-slave-t3q3"
server_key: "8a95676779daffd42febcb49cd6da221"
server_key_iv: "2d6c115c7677fd37a622d12160f6c5a0"
server_key_version: "encryptionkey@0"
I20260808 13:04:54.928846 18079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:54.945138 18079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:54.950062 18079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:54.950389 18079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:54.969045 18079 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:46009
I20260808 13:04:54.969080 18190 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:46009 every 8 connection(s)
I20260808 13:04:54.970484 18079 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:04:54.971923 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 18079
I20260808 13:04:54.972158 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:04:54.973045 1522 external_mini_cluster.cc:1428] Setting key a0bf4d4d53f0d5fe05c1e163e747880b
I20260808 13:04:54.975386 18192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 13:04:54 dist-test-slave-t3q3 krb5kdc[18056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:04:54.989216 18192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb: Bootstrap starting.
I20260808 13:04:54.992429 18192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:54.993723 18192 log.cc:824] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb: Log is configured to *not* fsync() on all Append() calls
I20260808 13:04:54.997063 18192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb: No bootstrap required, opened a new log
I20260808 13:04:54.997557 18196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:54.975130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41228 (local address 127.1.124.190:46009)
0808 13:04:54.975937 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:54.975950 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:54.975991 (+ 41us) server_negotiation.cc:407] Connection header received
0808 13:04:54.977706 (+ 1715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:54.977728 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:54.978110 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:54.978439 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:54.980365 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:54.981755 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:54.983532 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:54.984272 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:54.984399 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:54.989379 (+ 4980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:54.989392 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:54.989419 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:54.989476 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:54.992952 (+ 3476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:54.993943 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:54.993950 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:54.993957 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:54.994129 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:54.994597 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:54.994603 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:54.994609 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:54.995332 (+ 723us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:54.995558 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:54.996023 (+ 465us) server_negotiation.cc:299] Negotiation successful
0808 13:04:54.996525 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":381,"threads_started":1}
I20260808 13:04:55.004580 18192 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac63d5b391d44886a7440c696faba1fb" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 46009 } }
I20260808 13:04:55.004946 18192 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:55.005057 18192 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac63d5b391d44886a7440c696faba1fb, State: Initialized, Role: FOLLOWER
I20260808 13:04:55.005698 18192 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac63d5b391d44886a7440c696faba1fb" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 46009 } }
I20260808 13:04:55.005949 18192 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:04:55.006083 18192 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:04:55.006271 18192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:55.008569 18192 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac63d5b391d44886a7440c696faba1fb" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 46009 } }
I20260808 13:04:55.009104 18192 leader_election.cc:302] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac63d5b391d44886a7440c696faba1fb; no voters:
I20260808 13:04:55.009672 18192 leader_election.cc:288] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:04:55.009859 18199 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:04:55.010565 18199 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb [term 1 LEADER]: Becoming Leader. State: Replica: ac63d5b391d44886a7440c696faba1fb, State: Running, Role: LEADER
I20260808 13:04:55.011157 18199 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac63d5b391d44886a7440c696faba1fb" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 46009 } }
I20260808 13:04:55.011734 18192 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:04:55.013959 18204 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac63d5b391d44886a7440c696faba1fb. Latest consensus state: current_term: 1 leader_uuid: "ac63d5b391d44886a7440c696faba1fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac63d5b391d44886a7440c696faba1fb" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 46009 } } }
I20260808 13:04:55.014339 18204 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:55.015089 18202 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac63d5b391d44886a7440c696faba1fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac63d5b391d44886a7440c696faba1fb" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 46009 } } }
I20260808 13:04:55.015331 18213 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:04:55.016706 18202 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb [sys.catalog]: This master's current role is: LEADER
I20260808 13:04:55.021728 18213 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:04:55.029920 18213 catalog_manager.cc:1347] Generated new cluster ID: 72988546093e4da1853d53b3e6a61b80
I20260808 13:04:55.030056 18213 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:04:55.051589 18213 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:04:55.053011 18213 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:04:55.061565 18213 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb: Generated new TSK 0
I20260808 13:04:55.062405 18213 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.124.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
I20260808 13:04:55.139918 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:46009
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:41875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:55.491138 18235 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:55.491407 18235 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:55.491561 18235 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:55.499198 18235 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:55.499326 18235 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:55.499512 18235 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:04:55.509234 18235 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:41875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:46009
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:55.510525 18235 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:55.511976 18235 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:55.525609 18245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:55.525631 18247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:55.526139 18244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:55.526564 18235 server_base.cc:1034] running on GCE node
I20260808 13:04:55.527129 18235 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:55.528008 18235 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:55.529271 18235 hybrid_clock.cc:648] HybridClock initialized: now 1786194295529232 us; error 36 us; skew 500 ppm
I20260808 13:04:55.529601 18235 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:55 dist-test-slave-t3q3 krb5kdc[18056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194295, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:55.533082 18235 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:04:55.534946 18235 webserver.cc:460] Webserver started at http://127.1.124.129:43473/ using document root <none> and password file <none>
I20260808 13:04:55.535699 18235 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:55.535806 18235 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:55.536162 18235 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:55.539108 18235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "dc238c3e53ac40d68e7430f6e2f38402"
format_stamp: "Formatted at 2026-08-08 13:04:55 on dist-test-slave-t3q3"
server_key: "fbd40a44adc1cd6d9dedd4c49636d80d"
server_key_iv: "a9308e675880d831398899fc1b8dd639"
server_key_version: "encryptionkey@0"
I20260808 13:04:55.539860 18235 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "dc238c3e53ac40d68e7430f6e2f38402"
format_stamp: "Formatted at 2026-08-08 13:04:55 on dist-test-slave-t3q3"
server_key: "fbd40a44adc1cd6d9dedd4c49636d80d"
server_key_iv: "a9308e675880d831398899fc1b8dd639"
server_key_version: "encryptionkey@0"
I20260808 13:04:55.545027 18235 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.002s
I20260808 13:04:55.548417 18263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:55.549839 18235 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 13:04:55.549973 18235 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "dc238c3e53ac40d68e7430f6e2f38402"
format_stamp: "Formatted at 2026-08-08 13:04:55 on dist-test-slave-t3q3"
server_key: "fbd40a44adc1cd6d9dedd4c49636d80d"
server_key_iv: "a9308e675880d831398899fc1b8dd639"
server_key_version: "encryptionkey@0"
I20260808 13:04:55.550174 18235 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:55.569766 18235 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:55.573894 18235 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:55.574203 18235 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:55.575169 18235 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:55.576583 18235 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:55.576661 18235 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:55.576839 18235 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:55.576893 18235 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:55.610615 18235 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:36043
I20260808 13:04:55.610738 18453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:36043 every 8 connection(s)
I20260808 13:04:55.611994 18235 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:04:55.621280 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 18235
I20260808 13:04:55.621506 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:04:55.622362 1522 external_mini_cluster.cc:1428] Setting key d1fe206e87ebe747b7c7feeebc1cf227
Aug 08 13:04:55 dist-test-slave-t3q3 krb5kdc[18056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194295, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:04:55.635149 18458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:55.614716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:54085 (local address 127.1.124.190:46009)
0808 13:04:55.615181 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:55.615185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:55.616034 (+ 849us) server_negotiation.cc:407] Connection header received
0808 13:04:55.618027 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:55.618032 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:55.618125 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:55.618256 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:55.619786 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:55.620569 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:55.623074 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:55.623608 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:55.623727 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:55.627694 (+ 3967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:55.627704 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:55.627712 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:55.627761 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:55.631370 (+ 3609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:55.632250 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:55.632258 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:55.632264 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:55.632426 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:55.632884 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:55.632892 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:55.632898 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:55.633323 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:55.633481 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:55.634393 (+ 912us) server_negotiation.cc:299] Negotiation successful
0808 13:04:55.634666 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":253,"threads_started":1}
I20260808 13:04:55.635936 18460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:55.615131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:46009 (local address 127.1.124.129:54085)
0808 13:04:55.615837 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:55.615910 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:55.617720 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:55.618498 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:55.618506 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:55.619020 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:55.619593 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:55.619612 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:55.620741 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:55.620747 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:55.622871 (+ 2124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:55.622887 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:55.623865 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:55.623873 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:55.624197 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:55.624941 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:55.624969 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:55.627435 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:55.631522 (+ 4087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:55.631530 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:55.631544 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:55.632131 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:55.632556 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:55.632559 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:55.632561 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:55.632756 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:55.633534 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:55.633543 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:55.633957 (+ 414us) client_negotiation.cc:769] Sending connection context
0808 13:04:55.634229 (+ 272us) client_negotiation.cc:240] Negotiation successful
0808 13:04:55.634682 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":276,"threads_started":1}
I20260808 13:04:55.637739 18454 heartbeater.cc:347] Connected to a master server at 127.1.124.190:46009
I20260808 13:04:55.638160 18454 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:55.639182 18454 heartbeater.cc:511] Master 127.1.124.190:46009 requested a full tablet report, sending...
I20260808 13:04:55.641568 18142 ts_manager.cc:194] Registered new tserver with Master: dc238c3e53ac40d68e7430f6e2f38402 (127.1.124.129:36043)
I20260808 13:04:55.643038 18142 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:54085
WARNING: no policy specified for kudu/127.1.124.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
I20260808 13:04:55.692667 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:46009
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:41875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:56.040639 18463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:56.040935 18463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:56.041117 18463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:56.048584 18463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:56.048707 18463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:56.048897 18463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:04:56.059424 18463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:41875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:46009
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:56.061004 18463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:56.062433 18463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:56.076499 18473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:56.076576 18475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:56.076545 18478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:56.077296 18463 server_base.cc:1034] running on GCE node
I20260808 13:04:56.078019 18463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:56.079490 18463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:56.080744 18463 hybrid_clock.cc:648] HybridClock initialized: now 1786194296080655 us; error 94 us; skew 500 ppm
I20260808 13:04:56.081182 18463 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:56 dist-test-slave-t3q3 krb5kdc[18056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194296, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:56.085022 18463 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:04:56.087658 18463 webserver.cc:460] Webserver started at http://127.1.124.130:34115/ using document root <none> and password file <none>
I20260808 13:04:56.088784 18463 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:56.088949 18463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:56.089354 18463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:56.093186 18463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "fb43411cd2f74c548eb706bcdd4332fb"
format_stamp: "Formatted at 2026-08-08 13:04:56 on dist-test-slave-t3q3"
server_key: "661706709144f907aac0f153cc28b381"
server_key_iv: "b651af79e43644966f5a23ffd27353ed"
server_key_version: "encryptionkey@0"
I20260808 13:04:56.094399 18463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "fb43411cd2f74c548eb706bcdd4332fb"
format_stamp: "Formatted at 2026-08-08 13:04:56 on dist-test-slave-t3q3"
server_key: "661706709144f907aac0f153cc28b381"
server_key_iv: "b651af79e43644966f5a23ffd27353ed"
server_key_version: "encryptionkey@0"
I20260808 13:04:56.103953 18463 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.005s sys 0.004s
I20260808 13:04:56.109179 18486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:56.111222 18463 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260808 13:04:56.111447 18463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "fb43411cd2f74c548eb706bcdd4332fb"
format_stamp: "Formatted at 2026-08-08 13:04:56 on dist-test-slave-t3q3"
server_key: "661706709144f907aac0f153cc28b381"
server_key_iv: "b651af79e43644966f5a23ffd27353ed"
server_key_version: "encryptionkey@0"
I20260808 13:04:56.111698 18463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:56.127332 18463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:56.133335 18463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:56.133646 18463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:56.134949 18463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:56.136822 18463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:56.136927 18463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:56.137066 18463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:56.137143 18463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:56.172935 18463 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:40615
I20260808 13:04:56.172995 18678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:40615 every 8 connection(s)
I20260808 13:04:56.174316 18463 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:04:56.176558 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 18463
I20260808 13:04:56.176751 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:04:56.177368 1522 external_mini_cluster.cc:1428] Setting key 4c3d2c5abb6ed32d80eadb79e60299ab
Aug 08 13:04:56 dist-test-slave-t3q3 krb5kdc[18056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194296, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:04:56.196867 18681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:56.177104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:38567 (local address 127.1.124.190:46009)
0808 13:04:56.177621 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:56.177626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:56.178818 (+ 1192us) server_negotiation.cc:407] Connection header received
0808 13:04:56.180868 (+ 2050us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:56.180877 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:56.180969 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:56.181098 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:56.182608 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.183761 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.185391 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.186139 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.186299 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:56.189925 (+ 3626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:56.189934 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:56.189941 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:56.189988 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:56.193273 (+ 3285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:56.194270 (+ 997us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:56.194275 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:56.194277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:56.194408 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:56.194950 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:56.194953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:56.194955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:56.195244 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:56.195355 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:56.196398 (+ 1043us) server_negotiation.cc:299] Negotiation successful
0808 13:04:56.196576 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":276,"threads_started":1}
I20260808 13:04:56.198531 18684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:56.177726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:46009 (local address 127.1.124.130:38567)
0808 13:04:56.178630 (+ 904us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:56.178687 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:56.180575 (+ 1888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:56.181325 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:56.181333 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:56.181839 (+ 506us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:56.182427 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:56.182445 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.183909 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.183913 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:56.185236 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:56.185255 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.186375 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.186380 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:56.186587 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:56.187343 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:56.187366 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:56.189665 (+ 2299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:56.193425 (+ 3760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:56.193438 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:56.193459 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:56.194124 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:56.194520 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:56.194527 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:56.194534 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:56.194791 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:56.195377 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:56.195388 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:56.195922 (+ 534us) client_negotiation.cc:769] Sending connection context
0808 13:04:56.196305 (+ 383us) client_negotiation.cc:240] Negotiation successful
0808 13:04:56.196927 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":387,"threads_started":1}
I20260808 13:04:56.200176 18679 heartbeater.cc:347] Connected to a master server at 127.1.124.190:46009
I20260808 13:04:56.200510 18679 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:56.201251 18679 heartbeater.cc:511] Master 127.1.124.190:46009 requested a full tablet report, sending...
I20260808 13:04:56.202857 18142 ts_manager.cc:194] Registered new tserver with Master: fb43411cd2f74c548eb706bcdd4332fb (127.1.124.130:40615)
I20260808 13:04:56.203724 18142 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:38567
WARNING: no policy specified for kudu/127.1.124.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
I20260808 13:04:56.247833 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:46009
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:41875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:04:56.596830 18689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:04:56.597127 18689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:04:56.597244 18689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:04:56.604455 18689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:04:56.604565 18689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:04:56.604676 18689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:04:56.614607 18689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:41875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:46009
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:04:56.615861 18689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:04:56.617408 18689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:04:56.631520 18703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:56.631467 18700 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:04:56.631749 18701 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:04:56.632799 18689 server_base.cc:1034] running on GCE node
I20260808 13:04:56.633481 18689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:04:56.634508 18689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:04:56.635756 18689 hybrid_clock.cc:648] HybridClock initialized: now 1786194296635702 us; error 46 us; skew 500 ppm
I20260808 13:04:56.636149 18689 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:04:56 dist-test-slave-t3q3 krb5kdc[18056](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194296, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:04:56.639631 18689 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:04:56.641314 18689 webserver.cc:460] Webserver started at http://127.1.124.131:41553/ using document root <none> and password file <none>
I20260808 13:04:56.642047 18689 fs_manager.cc:359] Metadata directory not provided
I20260808 13:04:56.642211 18689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:04:56.642546 18689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:04:56.645627 18689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "2b94e06c8cbb4311a3c8530f568b26df"
format_stamp: "Formatted at 2026-08-08 13:04:56 on dist-test-slave-t3q3"
server_key: "874cfbd4e950957505721d3238e260d4"
server_key_iv: "c14e8599907f48c298cc37e8d468a4a7"
server_key_version: "encryptionkey@0"
I20260808 13:04:56.646456 18689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "2b94e06c8cbb4311a3c8530f568b26df"
format_stamp: "Formatted at 2026-08-08 13:04:56 on dist-test-slave-t3q3"
server_key: "874cfbd4e950957505721d3238e260d4"
server_key_iv: "c14e8599907f48c298cc37e8d468a4a7"
server_key_version: "encryptionkey@0"
I20260808 13:04:56.646898 18454 heartbeater.cc:503] Master 127.1.124.190:46009 was elected leader, sending a full tablet report...
I20260808 13:04:56.652292 18689 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 13:04:56.655848 18717 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:56.657150 18689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:04:56.657364 18689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "2b94e06c8cbb4311a3c8530f568b26df"
format_stamp: "Formatted at 2026-08-08 13:04:56 on dist-test-slave-t3q3"
server_key: "874cfbd4e950957505721d3238e260d4"
server_key_iv: "c14e8599907f48c298cc37e8d468a4a7"
server_key_version: "encryptionkey@0"
I20260808 13:04:56.657612 18689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:04:56.689396 18689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:04:56.693646 18689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:04:56.693975 18689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:04:56.695039 18689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:04:56.696532 18689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:04:56.696642 18689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:56.696806 18689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:04:56.696883 18689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:04:56.734117 18689 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:42351
I20260808 13:04:56.734138 18902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:42351 every 8 connection(s)
I20260808 13:04:56.736115 18689 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:04:56.740510 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 18689
I20260808 13:04:56.740696 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:04:56.741140 1522 external_mini_cluster.cc:1428] Setting key ad66d1fec37abf5f2f58371812c84afe
Aug 08 13:04:56 dist-test-slave-t3q3 krb5kdc[18056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194296, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:04:56.760445 18907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:56.739422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:43615 (local address 127.1.124.190:46009)
0808 13:04:56.739936 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:56.739942 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:56.742359 (+ 2417us) server_negotiation.cc:407] Connection header received
0808 13:04:56.744885 (+ 2526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:56.744889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:56.744958 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:56.745042 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:56.746612 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.747442 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.749159 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.749562 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.749694 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:56.753378 (+ 3684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:56.753383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:56.753387 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:56.753421 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:56.756602 (+ 3181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:56.757709 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:56.757713 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:56.757715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:56.757864 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:56.758393 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:56.758397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:56.758399 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:56.758792 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:56.758933 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:56.759788 (+ 855us) server_negotiation.cc:299] Negotiation successful
0808 13:04:56.759984 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":235,"threads_started":1}
I20260808 13:04:56.761587 18909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:56.740187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:46009 (local address 127.1.124.131:43615)
0808 13:04:56.742160 (+ 1973us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:56.742232 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:56.744546 (+ 2314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:56.745260 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:56.745269 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:56.745816 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:04:56.746437 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:56.746456 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.747596 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.747599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:56.749005 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:56.749019 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.749781 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.749787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:56.749989 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:56.750634 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:04:56.750657 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:04:56.753095 (+ 2438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:04:56.756787 (+ 3692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:56.756797 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:56.756817 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:56.757548 (+ 731us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:56.758007 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:04:56.758045 (+ 38us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:04:56.758047 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:04:56.758265 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:04:56.758969 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:04:56.758976 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:04:56.759369 (+ 393us) client_negotiation.cc:769] Sending connection context
0808 13:04:56.759658 (+ 289us) client_negotiation.cc:240] Negotiation successful
0808 13:04:56.760197 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1638,"thread_start_us":1475,"threads_started":1}
I20260808 13:04:56.763406 18903 heartbeater.cc:347] Connected to a master server at 127.1.124.190:46009
I20260808 13:04:56.763756 18903 heartbeater.cc:464] Registering TS with master...
I20260808 13:04:56.764631 18903 heartbeater.cc:511] Master 127.1.124.190:46009 requested a full tablet report, sending...
I20260808 13:04:56.766278 18142 ts_manager.cc:194] Registered new tserver with Master: 2b94e06c8cbb4311a3c8530f568b26df (127.1.124.131:42351)
I20260808 13:04:56.767148 18142 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:43615
I20260808 13:04:56.770190 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:04:56.788326 18907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:56.775058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41230 (local address 127.1.124.190:46009)
0808 13:04:56.775273 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:56.775278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:56.775886 (+ 608us) server_negotiation.cc:407] Connection header received
0808 13:04:56.775996 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:56.776000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:56.776135 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:56.776267 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:56.777867 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.778671 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.780049 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.780536 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.780637 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:56.782989 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:56.782994 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:56.782997 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:56.783030 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:56.785610 (+ 2580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:56.786310 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:56.786315 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:56.786317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:56.786447 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:56.786874 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:56.786878 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:56.786881 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:56.787287 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:56.787475 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:56.787814 (+ 339us) server_negotiation.cc:299] Negotiation successful
0808 13:04:56.787987 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 13:04:56.792490 18142 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41230:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:04:56.799409 18142 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:04:56.823777 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:56.815256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54360 (local address 127.1.124.131:42351)
0808 13:04:56.816310 (+ 1054us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:56.816319 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:56.816349 (+ 30us) server_negotiation.cc:407] Connection header received
0808 13:04:56.816443 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:56.816450 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:56.816648 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:56.817588 (+ 940us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:56.818001 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.819148 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.821549 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.822457 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.822624 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:56.822876 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:56.823233 (+ 357us) server_negotiation.cc:299] Negotiation successful
0808 13:04:56.823396 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":817,"thread_start_us":613,"threads_started":1}
I20260808 13:04:56.823802 18931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:56.814661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:42351 (local address 127.0.0.1:54360)
0808 13:04:56.815412 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:56.815445 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:56.815589 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:56.816939 (+ 1350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:56.816943 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:56.816959 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:56.817791 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:56.817810 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.819283 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.819291 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:56.821389 (+ 2098us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:56.821405 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.822714 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.822720 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:56.822958 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:56.823017 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 13:04:56.823163 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 13:04:56.823360 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"mutex_wait_us":83,"thread_start_us":918,"threads_started":1}
I20260808 13:04:56.827440 18786 tablet_service.cc:1467] Processing CreateTablet for tablet 140a22fd1d7e487fa66aac81d0a7fced (DEFAULT_TABLE table=test-table [id=63b4b68efc59454ebfc63fc69071be66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:04:56.828928 18925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:56.811916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:36043 (local address 127.0.0.1:56030)
0808 13:04:56.813592 (+ 1676us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:56.813637 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:56.813800 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:56.817264 (+ 3464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:56.817268 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:56.817293 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:56.818069 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:56.818097 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.819924 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.819932 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:56.822126 (+ 2194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:56.822141 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.827948 (+ 5807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.827960 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:56.828200 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:56.828237 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 13:04:56.828338 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 13:04:56.828472 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1447,"mutex_wait_us":28,"thread_start_us":1303,"threads_started":1}
I20260808 13:04:56.829243 18928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:56.812476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51014 (local address 127.1.124.130:40615)
0808 13:04:56.815026 (+ 2550us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:56.815035 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:56.816171 (+ 1136us) server_negotiation.cc:407] Connection header received
0808 13:04:56.816286 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:56.816297 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:56.816554 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:56.816852 (+ 298us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:56.818169 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.820492 (+ 2323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.823006 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.824737 (+ 1731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.828236 (+ 3499us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:56.828418 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:56.828716 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 13:04:56.828882 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2381,"thread_start_us":2239,"threads_started":1}
I20260808 13:04:56.829296 18786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 140a22fd1d7e487fa66aac81d0a7fced. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:56.829537 18927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:56.812305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56030 (local address 127.1.124.129:36043)
0808 13:04:56.816596 (+ 4291us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:56.816604 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:56.816631 (+ 27us) server_negotiation.cc:407] Connection header received
0808 13:04:56.816725 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:56.816732 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:56.816949 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:56.817235 (+ 286us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:56.818309 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.819788 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.822314 (+ 2526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.824839 (+ 2525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.825084 (+ 245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:56.825256 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:56.829034 (+ 3778us) server_negotiation.cc:299] Negotiation successful
0808 13:04:56.829203 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4125,"thread_start_us":3975,"threads_started":1}
I20260808 13:04:56.829769 18929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:56.812723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:40615 (local address 127.0.0.1:51014)
0808 13:04:56.814379 (+ 1656us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:56.814395 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:56.814509 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:56.816899 (+ 2390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:56.816909 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:56.816942 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:56.817894 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:56.817913 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.820647 (+ 2734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.820653 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:56.822867 (+ 2214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:56.822883 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.827884 (+ 5001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.827897 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:56.828127 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:56.828552 (+ 425us) client_negotiation.cc:769] Sending connection context
0808 13:04:56.828674 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 13:04:56.828817 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1534,"mutex_wait_us":417,"thread_start_us":1355,"threads_started":1}
I20260808 13:04:56.833025 18334 tablet_service.cc:1467] Processing CreateTablet for tablet 140a22fd1d7e487fa66aac81d0a7fced (DEFAULT_TABLE table=test-table [id=63b4b68efc59454ebfc63fc69071be66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:04:56.833303 18572 tablet_service.cc:1467] Processing CreateTablet for tablet 140a22fd1d7e487fa66aac81d0a7fced (DEFAULT_TABLE table=test-table [id=63b4b68efc59454ebfc63fc69071be66]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:04:56.834517 18334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 140a22fd1d7e487fa66aac81d0a7fced. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:56.834581 18572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 140a22fd1d7e487fa66aac81d0a7fced. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:04:56.844493 18934 tablet_bootstrap.cc:492] T 140a22fd1d7e487fa66aac81d0a7fced P 2b94e06c8cbb4311a3c8530f568b26df: Bootstrap starting.
I20260808 13:04:56.846372 18937 tablet_bootstrap.cc:492] T 140a22fd1d7e487fa66aac81d0a7fced P dc238c3e53ac40d68e7430f6e2f38402: Bootstrap starting.
I20260808 13:04:56.848361 18934 tablet_bootstrap.cc:654] T 140a22fd1d7e487fa66aac81d0a7fced P 2b94e06c8cbb4311a3c8530f568b26df: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:56.848876 18938 tablet_bootstrap.cc:492] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb: Bootstrap starting.
I20260808 13:04:56.850124 18937 tablet_bootstrap.cc:654] T 140a22fd1d7e487fa66aac81d0a7fced P dc238c3e53ac40d68e7430f6e2f38402: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:56.850754 18934 log.cc:824] T 140a22fd1d7e487fa66aac81d0a7fced P 2b94e06c8cbb4311a3c8530f568b26df: Log is configured to *not* fsync() on all Append() calls
I20260808 13:04:56.851402 18937 log.cc:824] T 140a22fd1d7e487fa66aac81d0a7fced P dc238c3e53ac40d68e7430f6e2f38402: Log is configured to *not* fsync() on all Append() calls
I20260808 13:04:56.852816 18938 tablet_bootstrap.cc:654] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb: Neither blocks nor log segments found. Creating new log.
I20260808 13:04:56.853847 18934 tablet_bootstrap.cc:492] T 140a22fd1d7e487fa66aac81d0a7fced P 2b94e06c8cbb4311a3c8530f568b26df: No bootstrap required, opened a new log
I20260808 13:04:56.854118 18938 log.cc:824] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb: Log is configured to *not* fsync() on all Append() calls
I20260808 13:04:56.854172 18934 ts_tablet_manager.cc:1397] T 140a22fd1d7e487fa66aac81d0a7fced P 2b94e06c8cbb4311a3c8530f568b26df: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.003s
I20260808 13:04:56.854511 18937 tablet_bootstrap.cc:492] T 140a22fd1d7e487fa66aac81d0a7fced P dc238c3e53ac40d68e7430f6e2f38402: No bootstrap required, opened a new log
I20260808 13:04:56.854816 18937 ts_tablet_manager.cc:1397] T 140a22fd1d7e487fa66aac81d0a7fced P dc238c3e53ac40d68e7430f6e2f38402: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.006s
I20260808 13:04:56.858733 18938 tablet_bootstrap.cc:492] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb: No bootstrap required, opened a new log
I20260808 13:04:56.858990 18938 ts_tablet_manager.cc:1397] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.006s
I20260808 13:04:56.862561 18934 raft_consensus.cc:348] T 140a22fd1d7e487fa66aac81d0a7fced P 2b94e06c8cbb4311a3c8530f568b26df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc238c3e53ac40d68e7430f6e2f38402" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 36043 } } peers { permanent_uuid: "fb43411cd2f74c548eb706bcdd4332fb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40615 } } peers { permanent_uuid: "2b94e06c8cbb4311a3c8530f568b26df" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42351 } }
I20260808 13:04:56.862929 18934 raft_consensus.cc:374] T 140a22fd1d7e487fa66aac81d0a7fced P 2b94e06c8cbb4311a3c8530f568b26df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:56.863054 18934 raft_consensus.cc:729] T 140a22fd1d7e487fa66aac81d0a7fced P 2b94e06c8cbb4311a3c8530f568b26df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b94e06c8cbb4311a3c8530f568b26df, State: Initialized, Role: FOLLOWER
I20260808 13:04:56.863670 18934 consensus_queue.cc:260] T 140a22fd1d7e487fa66aac81d0a7fced P 2b94e06c8cbb4311a3c8530f568b26df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc238c3e53ac40d68e7430f6e2f38402" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 36043 } } peers { permanent_uuid: "fb43411cd2f74c548eb706bcdd4332fb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40615 } } peers { permanent_uuid: "2b94e06c8cbb4311a3c8530f568b26df" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42351 } }
I20260808 13:04:56.864804 18903 heartbeater.cc:503] Master 127.1.124.190:46009 was elected leader, sending a full tablet report...
I20260808 13:04:56.865568 18934 ts_tablet_manager.cc:1428] T 140a22fd1d7e487fa66aac81d0a7fced P 2b94e06c8cbb4311a3c8530f568b26df: Time spent starting tablet: real 0.011s user 0.005s sys 0.007s
I20260808 13:04:56.865345 18937 raft_consensus.cc:348] T 140a22fd1d7e487fa66aac81d0a7fced P dc238c3e53ac40d68e7430f6e2f38402 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc238c3e53ac40d68e7430f6e2f38402" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 36043 } } peers { permanent_uuid: "fb43411cd2f74c548eb706bcdd4332fb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40615 } } peers { permanent_uuid: "2b94e06c8cbb4311a3c8530f568b26df" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42351 } }
I20260808 13:04:56.865767 18937 raft_consensus.cc:374] T 140a22fd1d7e487fa66aac81d0a7fced P dc238c3e53ac40d68e7430f6e2f38402 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:56.865852 18937 raft_consensus.cc:729] T 140a22fd1d7e487fa66aac81d0a7fced P dc238c3e53ac40d68e7430f6e2f38402 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc238c3e53ac40d68e7430f6e2f38402, State: Initialized, Role: FOLLOWER
I20260808 13:04:56.866549 18937 consensus_queue.cc:260] T 140a22fd1d7e487fa66aac81d0a7fced P dc238c3e53ac40d68e7430f6e2f38402 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc238c3e53ac40d68e7430f6e2f38402" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 36043 } } peers { permanent_uuid: "fb43411cd2f74c548eb706bcdd4332fb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40615 } } peers { permanent_uuid: "2b94e06c8cbb4311a3c8530f568b26df" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42351 } }
I20260808 13:04:56.868273 18937 ts_tablet_manager.cc:1428] T 140a22fd1d7e487fa66aac81d0a7fced P dc238c3e53ac40d68e7430f6e2f38402: Time spent starting tablet: real 0.013s user 0.006s sys 0.010s
I20260808 13:04:56.869501 18938 raft_consensus.cc:348] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc238c3e53ac40d68e7430f6e2f38402" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 36043 } } peers { permanent_uuid: "fb43411cd2f74c548eb706bcdd4332fb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40615 } } peers { permanent_uuid: "2b94e06c8cbb4311a3c8530f568b26df" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42351 } }
I20260808 13:04:56.869946 18938 raft_consensus.cc:374] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:04:56.870064 18938 raft_consensus.cc:729] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb43411cd2f74c548eb706bcdd4332fb, State: Initialized, Role: FOLLOWER
I20260808 13:04:56.870628 18938 consensus_queue.cc:260] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc238c3e53ac40d68e7430f6e2f38402" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 36043 } } peers { permanent_uuid: "fb43411cd2f74c548eb706bcdd4332fb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40615 } } peers { permanent_uuid: "2b94e06c8cbb4311a3c8530f568b26df" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42351 } }
I20260808 13:04:56.871788 18679 heartbeater.cc:503] Master 127.1.124.190:46009 was elected leader, sending a full tablet report...
I20260808 13:04:56.872361 18938 ts_tablet_manager.cc:1428] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb: Time spent starting tablet: real 0.013s user 0.012s sys 0.003s
W20260808 13:04:56.926417 18680 tablet.cc:2369] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:04:56.976200 18946 raft_consensus.cc:482] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:04:56.976548 18946 raft_consensus.cc:504] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc238c3e53ac40d68e7430f6e2f38402" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 36043 } } peers { permanent_uuid: "fb43411cd2f74c548eb706bcdd4332fb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40615 } } peers { permanent_uuid: "2b94e06c8cbb4311a3c8530f568b26df" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42351 } }
I20260808 13:04:56.978575 18946 leader_election.cc:288] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dc238c3e53ac40d68e7430f6e2f38402 (127.1.124.129:36043), 2b94e06c8cbb4311a3c8530f568b26df (127.1.124.131:42351)
I20260808 13:04:56.986598 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:56.980310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:37743 (local address 127.1.124.131:42351)
0808 13:04:56.980502 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:56.980508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:56.980889 (+ 381us) server_negotiation.cc:407] Connection header received
0808 13:04:56.981069 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:56.981074 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:56.981150 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:56.981254 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:56.981899 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.982913 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.984823 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.985572 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.985742 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:56.985830 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:56.986206 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 13:04:56.986337 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 13:04:56.986644 18927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:56.979216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:33079 (local address 127.1.124.129:36043)
0808 13:04:56.979442 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:56.979449 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:56.980030 (+ 581us) server_negotiation.cc:407] Connection header received
0808 13:04:56.980406 (+ 376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:56.980411 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:56.980491 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:56.980559 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:04:56.981507 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.982601 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.984781 (+ 2180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.985586 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.985777 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:56.985853 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:56.986258 (+ 405us) server_negotiation.cc:299] Negotiation successful
0808 13:04:56.986387 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 13:04:56.986770 18948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:56.980032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:42351 (local address 127.1.124.130:37743)
0808 13:04:56.980759 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:56.980778 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:56.980921 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:56.981293 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:56.981298 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:56.981322 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:56.981723 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:56.981733 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.983091 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.983097 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:56.984623 (+ 1526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:56.984635 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.985811 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.985821 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:56.986030 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:56.986067 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 13:04:56.986141 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 13:04:56.986288 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"mutex_wait_us":26,"thread_start_us":453,"threads_started":1}
I20260808 13:04:56.986795 18947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:56.979112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:36043 (local address 127.1.124.130:33079)
0808 13:04:56.979921 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0808 13:04:56.979947 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 13:04:56.980179 (+ 232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:04:56.980778 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:04:56.980781 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:04:56.980802 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:04:56.981303 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:56.981319 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.982737 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.982742 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:56.984614 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:04:56.984627 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:04:56.985838 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:04:56.985847 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:04:56.986066 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:56.986089 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 13:04:56.986208 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 13:04:56.986331 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":67,"spinlock_wait_cycles":26368,"thread_start_us":438,"threads_started":1}
I20260808 13:04:56.987934 18374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "140a22fd1d7e487fa66aac81d0a7fced" candidate_uuid: "fb43411cd2f74c548eb706bcdd4332fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc238c3e53ac40d68e7430f6e2f38402" is_pre_election: true
I20260808 13:04:56.987934 18829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "140a22fd1d7e487fa66aac81d0a7fced" candidate_uuid: "fb43411cd2f74c548eb706bcdd4332fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b94e06c8cbb4311a3c8530f568b26df" is_pre_election: true
W20260808 13:04:56.988250 18906 tablet.cc:2369] T 140a22fd1d7e487fa66aac81d0a7fced P 2b94e06c8cbb4311a3c8530f568b26df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:04:56.988620 18829 raft_consensus.cc:2454] T 140a22fd1d7e487fa66aac81d0a7fced P 2b94e06c8cbb4311a3c8530f568b26df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb43411cd2f74c548eb706bcdd4332fb in term 0.
I20260808 13:04:56.988603 18374 raft_consensus.cc:2454] T 140a22fd1d7e487fa66aac81d0a7fced P dc238c3e53ac40d68e7430f6e2f38402 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb43411cd2f74c548eb706bcdd4332fb in term 0.
I20260808 13:04:56.989333 18489 leader_election.cc:302] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc238c3e53ac40d68e7430f6e2f38402, fb43411cd2f74c548eb706bcdd4332fb; no voters:
I20260808 13:04:56.989707 18946 raft_consensus.cc:2781] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:04:56.989861 18946 raft_consensus.cc:482] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:04:56.989980 18946 raft_consensus.cc:3037] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:56.992327 18946 raft_consensus.cc:504] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc238c3e53ac40d68e7430f6e2f38402" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 36043 } } peers { permanent_uuid: "fb43411cd2f74c548eb706bcdd4332fb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40615 } } peers { permanent_uuid: "2b94e06c8cbb4311a3c8530f568b26df" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42351 } }
I20260808 13:04:56.992918 18946 leader_election.cc:288] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb [CANDIDATE]: Term 1 election: Requested vote from peers dc238c3e53ac40d68e7430f6e2f38402 (127.1.124.129:36043), 2b94e06c8cbb4311a3c8530f568b26df (127.1.124.131:42351)
I20260808 13:04:56.993445 18374 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "140a22fd1d7e487fa66aac81d0a7fced" candidate_uuid: "fb43411cd2f74c548eb706bcdd4332fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc238c3e53ac40d68e7430f6e2f38402"
I20260808 13:04:56.993526 18829 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "140a22fd1d7e487fa66aac81d0a7fced" candidate_uuid: "fb43411cd2f74c548eb706bcdd4332fb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b94e06c8cbb4311a3c8530f568b26df"
I20260808 13:04:56.993677 18374 raft_consensus.cc:3037] T 140a22fd1d7e487fa66aac81d0a7fced P dc238c3e53ac40d68e7430f6e2f38402 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:56.993724 18829 raft_consensus.cc:3037] T 140a22fd1d7e487fa66aac81d0a7fced P 2b94e06c8cbb4311a3c8530f568b26df [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:04:56.996986 18829 raft_consensus.cc:2454] T 140a22fd1d7e487fa66aac81d0a7fced P 2b94e06c8cbb4311a3c8530f568b26df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb43411cd2f74c548eb706bcdd4332fb in term 1.
I20260808 13:04:56.996986 18374 raft_consensus.cc:2454] T 140a22fd1d7e487fa66aac81d0a7fced P dc238c3e53ac40d68e7430f6e2f38402 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb43411cd2f74c548eb706bcdd4332fb in term 1.
I20260808 13:04:56.997417 18489 leader_election.cc:302] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: dc238c3e53ac40d68e7430f6e2f38402, fb43411cd2f74c548eb706bcdd4332fb; no voters:
I20260808 13:04:56.997705 18946 raft_consensus.cc:2781] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:04:56.998138 18946 raft_consensus.cc:686] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb [term 1 LEADER]: Becoming Leader. State: Replica: fb43411cd2f74c548eb706bcdd4332fb, State: Running, Role: LEADER
I20260808 13:04:56.998606 18946 consensus_queue.cc:237] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc238c3e53ac40d68e7430f6e2f38402" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 36043 } } peers { permanent_uuid: "fb43411cd2f74c548eb706bcdd4332fb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40615 } } peers { permanent_uuid: "2b94e06c8cbb4311a3c8530f568b26df" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42351 } }
I20260808 13:04:57.003425 18142 catalog_manager.cc:5495] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb reported cstate change: term changed from 0 to 1, leader changed from <none> to fb43411cd2f74c548eb706bcdd4332fb (127.1.124.130). New cstate: current_term: 1 leader_uuid: "fb43411cd2f74c548eb706bcdd4332fb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc238c3e53ac40d68e7430f6e2f38402" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 36043 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb43411cd2f74c548eb706bcdd4332fb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 40615 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2b94e06c8cbb4311a3c8530f568b26df" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42351 } health_report { overall_health: UNKNOWN } } }
I20260808 13:04:57.061074 18928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:57.055424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51022 (local address 127.1.124.130:40615)
0808 13:04:57.055679 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:57.055687 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:57.055712 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:04:57.056001 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:57.056006 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:57.056141 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:57.056301 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:04:57.056689 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:57.057409 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:57.058752 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:57.059281 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:57.059416 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:57.059910 (+ 494us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:04:57.060472 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:04:57.060650 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:57.060708 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 13:04:57.060789 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260808 13:04:57.069533 18829 raft_consensus.cc:1261] T 140a22fd1d7e487fa66aac81d0a7fced P 2b94e06c8cbb4311a3c8530f568b26df [term 1 FOLLOWER]: Refusing update from remote peer fb43411cd2f74c548eb706bcdd4332fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:04:57.069542 18374 raft_consensus.cc:1261] T 140a22fd1d7e487fa66aac81d0a7fced P dc238c3e53ac40d68e7430f6e2f38402 [term 1 FOLLOWER]: Refusing update from remote peer fb43411cd2f74c548eb706bcdd4332fb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:04:57.070524 18952 consensus_queue.cc:1035] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b94e06c8cbb4311a3c8530f568b26df" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 42351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:04:57.070930 18946 consensus_queue.cc:1035] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc238c3e53ac40d68e7430f6e2f38402" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 36043 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:04:57.084739 18958 mvcc.cc:205] Tried to move back new op lower bound from 7316251840786771968 to 7316251840510382080. Current Snapshot: MvccSnapshot[applied={T|T < 7316251840786771968}]
I20260808 13:04:57.099169 18142 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:04:57.099440 18142 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41230:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:04:57.103379 18142 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac63d5b391d44886a7440c696faba1fb: Sending DeleteTablet for 3 replicas of tablet 140a22fd1d7e487fa66aac81d0a7fced
I20260808 13:04:57.105014 18572 tablet_service.cc:1514] Processing DeleteTablet for tablet 140a22fd1d7e487fa66aac81d0a7fced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:04:57 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:51014
I20260808 13:04:57.105052 18334 tablet_service.cc:1514] Processing DeleteTablet for tablet 140a22fd1d7e487fa66aac81d0a7fced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:04:57 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:56030
I20260808 13:04:57.105157 18786 tablet_service.cc:1514] Processing DeleteTablet for tablet 140a22fd1d7e487fa66aac81d0a7fced with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:04:57 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:54360
I20260808 13:04:57.107298 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 18235
I20260808 13:04:57.118045 18973 tablet_replica.cc:330] T 140a22fd1d7e487fa66aac81d0a7fced P 2b94e06c8cbb4311a3c8530f568b26df: stopping tablet replica
I20260808 13:04:57.118930 18973 raft_consensus.cc:2229] T 140a22fd1d7e487fa66aac81d0a7fced P 2b94e06c8cbb4311a3c8530f568b26df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:04:57.119746 18973 raft_consensus.cc:2258] T 140a22fd1d7e487fa66aac81d0a7fced P 2b94e06c8cbb4311a3c8530f568b26df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:04:57.121212 18974 tablet_replica.cc:330] T 140a22fd1d7e487fa66aac81d0a7fced P dc238c3e53ac40d68e7430f6e2f38402: stopping tablet replica
I20260808 13:04:57.121666 18974 raft_consensus.cc:2229] T 140a22fd1d7e487fa66aac81d0a7fced P dc238c3e53ac40d68e7430f6e2f38402 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:04:57.122181 18974 raft_consensus.cc:2258] T 140a22fd1d7e487fa66aac81d0a7fced P dc238c3e53ac40d68e7430f6e2f38402 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:04:57.122686 18973 ts_tablet_manager.cc:1905] T 140a22fd1d7e487fa66aac81d0a7fced P 2b94e06c8cbb4311a3c8530f568b26df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:57.123834 18972 tablet_replica.cc:330] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb: stopping tablet replica
I20260808 13:04:57.124276 18972 raft_consensus.cc:2229] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:04:57.124956 18972 raft_consensus.cc:2258] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:04:57.126679 18974 ts_tablet_manager.cc:1905] T 140a22fd1d7e487fa66aac81d0a7fced P dc238c3e53ac40d68e7430f6e2f38402: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:57.130432 18973 ts_tablet_manager.cc:1918] T 140a22fd1d7e487fa66aac81d0a7fced P 2b94e06c8cbb4311a3c8530f568b26df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:04:57.130690 18973 log.cc:1196] T 140a22fd1d7e487fa66aac81d0a7fced P 2b94e06c8cbb4311a3c8530f568b26df: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/140a22fd1d7e487fa66aac81d0a7fced
I20260808 13:04:57.131397 18973 ts_tablet_manager.cc:1939] T 140a22fd1d7e487fa66aac81d0a7fced P 2b94e06c8cbb4311a3c8530f568b26df: Deleting consensus metadata
I20260808 13:04:57.133630 18115 catalog_manager.cc:4841] TS 2b94e06c8cbb4311a3c8530f568b26df (127.1.124.131:42351): tablet 140a22fd1d7e487fa66aac81d0a7fced (table test-table [id=63b4b68efc59454ebfc63fc69071be66]) successfully deleted
I20260808 13:04:57.134420 18972 ts_tablet_manager.cc:1905] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:04:57.134548 18974 ts_tablet_manager.cc:1918] T 140a22fd1d7e487fa66aac81d0a7fced P dc238c3e53ac40d68e7430f6e2f38402: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:04:57.134737 18974 log.cc:1196] T 140a22fd1d7e487fa66aac81d0a7fced P dc238c3e53ac40d68e7430f6e2f38402: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/140a22fd1d7e487fa66aac81d0a7fced
I20260808 13:04:57.135361 18974 ts_tablet_manager.cc:1939] T 140a22fd1d7e487fa66aac81d0a7fced P dc238c3e53ac40d68e7430f6e2f38402: Deleting consensus metadata
Aug 08 13:04:57 dist-test-slave-t3q3 krb5kdc[18056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.129@KRBTEST.COM
I20260808 13:04:57.137348 18115 catalog_manager.cc:4841] TS dc238c3e53ac40d68e7430f6e2f38402 (127.1.124.129:36043): tablet 140a22fd1d7e487fa66aac81d0a7fced (table test-table [id=63b4b68efc59454ebfc63fc69071be66]) successfully deleted
I20260808 13:04:57.141945 18972 ts_tablet_manager.cc:1918] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:04:57.142172 18972 log.cc:1196] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/140a22fd1d7e487fa66aac81d0a7fced
I20260808 13:04:57.142731 18972 ts_tablet_manager.cc:1939] T 140a22fd1d7e487fa66aac81d0a7fced P fb43411cd2f74c548eb706bcdd4332fb: Deleting consensus metadata
I20260808 13:04:57.144250 18927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:57.109099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56040 (local address 127.1.124.129:36043)
0808 13:04:57.109287 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:57.109294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:57.123094 (+ 13800us) server_negotiation.cc:407] Connection header received
0808 13:04:57.123338 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:57.123343 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:57.123441 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:57.123606 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:57.125809 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:57.127010 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:57.129247 (+ 2237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:57.129922 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:57.130030 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:57.137959 (+ 7929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:57.137971 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:57.137985 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:57.138021 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:57.141068 (+ 3047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:57.141808 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:57.141815 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:57.141817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:57.141949 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:57.142538 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:57.142545 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:57.142552 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:57.142964 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:57.143282 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:57.143585 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 13:04:57.143821 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 13:04:57.144541 18111 catalog_manager.cc:4841] TS fb43411cd2f74c548eb706bcdd4332fb (127.1.124.130:40615): tablet 140a22fd1d7e487fa66aac81d0a7fced (table test-table [id=63b4b68efc59454ebfc63fc69071be66]) successfully deleted
I20260808 13:04:57.145293 18443 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:57.271726 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 18235
I20260808 13:04:57.301910 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 18463
Aug 08 13:04:57 dist-test-slave-t3q3 krb5kdc[18056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.130@KRBTEST.COM
I20260808 13:04:57.321319 18928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:57.302798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51038 (local address 127.1.124.130:40615)
0808 13:04:57.302972 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:57.302976 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:57.303040 (+ 64us) server_negotiation.cc:407] Connection header received
0808 13:04:57.303251 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:57.303255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:57.303365 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:57.303453 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:57.305240 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:57.306073 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:57.308271 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:57.308731 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:57.308826 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:57.314128 (+ 5302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:57.314143 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:57.314160 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:57.314208 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:57.318281 (+ 4073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:57.319126 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:57.319134 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:57.319136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:57.319307 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:57.319730 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:57.319733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:57.319736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:57.320177 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:57.320459 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:57.320783 (+ 324us) server_negotiation.cc:299] Negotiation successful
0808 13:04:57.320954 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 13:04:57.322077 18674 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:57.436754 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 18463
I20260808 13:04:57.464640 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 18689
Aug 08 13:04:57 dist-test-slave-t3q3 krb5kdc[18056](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.131@KRBTEST.COM
I20260808 13:04:57.481231 18932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:04:57.465464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54376 (local address 127.1.124.131:42351)
0808 13:04:57.465658 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 13:04:57.465672 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 13:04:57.465772 (+ 100us) server_negotiation.cc:407] Connection header received
0808 13:04:57.465997 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:04:57.466008 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:04:57.466075 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:04:57.466183 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:04:57.467519 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:57.468331 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:57.469610 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:04:57.470017 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:04:57.470157 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:04:57.475054 (+ 4897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:04:57.475072 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:04:57.475090 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:04:57.475141 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:04:57.478075 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:57.478870 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:57.478878 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:57.478881 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:57.479069 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:04:57.479484 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:04:57.479490 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:04:57.479493 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:04:57.479878 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:04:57.480250 (+ 372us) server_negotiation.cc:1035] Waiting for connection context
0808 13:04:57.480598 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 13:04:57.480840 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 13:04:57.481938 18896 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:57.575191 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 18689
I20260808 13:04:57.602716 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 18079
I20260808 13:04:57.604013 18185 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:04:57.713085 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 18079
2026-08-08T13:04:57Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4883 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:04:57 dist-test-slave-t3q3 krb5kdc[18985](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:04:57 dist-test-slave-t3q3 krb5kdc[18985](info): set up 2 sockets
Aug 08 13:04:57 dist-test-slave-t3q3 krb5kdc[18985](info): commencing operation
krb5kdc: starting...
W20260808 13:04:59.072808 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.309s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:04:59 dist-test-slave-t3q3 krb5kdc[18985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:04:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:04:59Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.124.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.124.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.124.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/rangeradmin_127.1.124.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.124.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/rangeradmin_127.1.124.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.124.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.124.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.124.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/rangerlookup_127.1.124.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.124.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/rangerlookup_127.1.124.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.124.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.124.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.124.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/HTTP_127.1.124.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.124.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/HTTP_127.1.124.148@KRBTEST.COM.keytab.
I20260808 13:04:59.493320 1522 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task9J1ueI/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/postgres -l logfile start
2026-08-08 13:05:02.131 UTC [19037] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 13:05:02.132 UTC [19037] LOG: listening on IPv4 address "127.1.124.148", port 48941
2026-08-08 13:05:02.135 UTC [19037] LOG: listening on Unix socket "/tmp/.s.PGSQL.48941"
2026-08-08 13:05:02.161 UTC [19040] LOG: database system was shut down at 2026-08-08 13:05:00 UTC
2026-08-08 13:05:02.165 UTC [19037] LOG: database system is ready to accept connections
I20260808 13:05:03.376413 1522 mini_postgres.cc:96] Postgres bound to 48941
2026-08-08 13:05:03.404 UTC [19056] FATAL: database "slave" does not exist
127.1.124.148:48941 - accepting connections
I20260808 13:05:03.405786 1522 mini_ranger.cc:161] Starting Ranger...
I20260808 13:05:03.442770 1522 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 13:05:03.991441 1522 mini_ranger.cc:87] Created ranger Postgres database
I20260808 13:05:03.991643 1522 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ranger-admin
2026-08-08 13:05:04,391 [I] DB FLAVOR :POSTGRES
2026-08-08 13:05:04,391 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 13:05:04,392 [I] Checking connection..
2026-08-08 13:05:04,392 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 13:05:04,668 [I] Checking connection passed.
2026-08-08 13:05:04,668 [I] --------- Verifying version history table ---------
2026-08-08 13:05:04,668 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 13:05:04,938 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 13:05:04,938 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 13:05:05,213 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 13:05:05,214 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 13:05:05,214 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 13:05:05.581 UTC [19167] WARNING: there is no transaction in progress
2026-08-08 13:05:05,598 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 13:05:05,599 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 13:05:06,169 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 13:05:06,170 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 13:05:06,171 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 13:05:06,674 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 13:05:07,146 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-t3q3.c.gcp-upstream.internal','N') ;"
2026-08-08 13:05:07,656 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 13:05:08,160 [I] Table x_portal_user does not exist in database ranger
2026-08-08 13:05:08,160 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 13:05:08,426 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 13:05:08,426 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 13:05:08,687 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 13:05:08,948 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 13:05:08,948 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 13:05:09.968 UTC [19360] WARNING: there is no transaction in progress
2026-08-08 13:05:09.985 UTC [19360] WARNING: there is no transaction in progress
2026-08-08 13:05:10.002 UTC [19360] WARNING: there is no transaction in progress
2026-08-08 13:05:10.019 UTC [19360] WARNING: there is no transaction in progress
2026-08-08 13:05:10.039 UTC [19360] WARNING: there is no transaction in progress
2026-08-08 13:05:10.058 UTC [19360] WARNING: there is no transaction in progress
2026-08-08 13:05:10.116 UTC [19360] WARNING: there is no transaction in progress
2026-08-08 13:05:10.122 UTC [19360] WARNING: there is no transaction in progress
2026-08-08 13:05:10.137 UTC [19360] WARNING: there is no transaction in progress
2026-08-08 13:05:10.147 UTC [19360] WARNING: there is no transaction in progress
2026-08-08 13:05:10.158 UTC [19360] WARNING: there is no transaction in progress
2026-08-08 13:05:10.171 UTC [19360] WARNING: there is no transaction in progress
2026-08-08 13:05:10.180 UTC [19360] WARNING: there is no transaction in progress
2026-08-08 13:05:10.187 UTC [19360] WARNING: there is no transaction in progress
2026-08-08 13:05:10.194 UTC [19360] WARNING: there is no transaction in progress
2026-08-08 13:05:10.202 UTC [19360] WARNING: there is no transaction in progress
2026-08-08 13:05:10,863 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 13:05:10,864 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 13:05:11,114 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 13:05:11,114 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 13:05:11,386 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 13:05:11,386 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 13:05:11,643 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 13:05:11,643 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 13:05:11,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 13:05:12,166 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-t3q3.c.gcp-upstream.internal';"
2026-08-08 13:05:12,430 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 13:05:12,430 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 13:05:12,708 [I] DB_PATCHES have already been applied
I20260808 13:05:12.714977 1522 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ranger-admin:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task9J1ueI/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 13:05:12.715135 1522 mini_ranger.cc:193] Using host: 127.1.124.148
I20260808 13:05:12.741520 1522 mini_ranger.cc:227] Ranger admin URL: http://127.1.124.148:36211
Aug 08, 2026 1:05:13 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 1:05:13 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 1:05:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 1:05:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 1:05:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 1:05:13 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 1:05:13 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 13:05:14.212718 1522 mini_ranger.cc:160] Time spent starting Ranger: real 10.807s user 0.001s sys 0.074s
Aug 08, 2026 1:05:35 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 1:05:35 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 1:05:35 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 1:05:35 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 1:05:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 1:05:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 1:05:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 1:05:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 1:05:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 1:05:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 1:05:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 1:05:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 1:05:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 1:05:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 1:05:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 1:05:35 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 1:05:35 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 1:05:36 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 13:05:36.681710 1522 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.124.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.124.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.124.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/rangerkms_127.1.124.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.124.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/rangerkms_127.1.124.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.124.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.124.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.124.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/HTTP_127.1.124.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.124.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/HTTP_127.1.124.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 13:05:36 dist-test-slave-t3q3 krb5kdc[18985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194336, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 13:05:36.903752 1522 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 13:05:36.933652 1522 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 13:05:37.839586 1522 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 13:05:37.839723 1522 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ranger-kms
I20260808 13:05:37.839913 1522 mini_ranger_kms.cc:227] Using postgres at 127.1.124.148:48941
2026-08-08 13:05:38,725 [I] DB FLAVOR :POSTGRES
2026-08-08 13:05:38,726 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 13:05:38,726 [I] Checking connection
2026-08-08 13:05:38,726 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 13:05:39,007 [I] connection success
2026-08-08 13:05:39,008 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 13:05:39,008 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 13:05:39,008 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 13:05:39,260 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 13:05:39,260 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 13:05:39,260 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 13:05:39,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:48941/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 13:05:39,530 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 13:05:40.055212 1522 mini_ranger_kms.cc:310] Created kms service
I20260808 13:05:40.251343 1522 mini_ranger_kms.cc:326] Created kudu user
I20260808 13:05:40.299050 1522 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 13:05:40 dist-test-slave-t3q3 krb5kdc[18985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194336, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
I20260808 13:05:40.714576 1522 mini_ranger_kms.cc:384] Added ranger policy
I20260808 13:05:40.716163 1522 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ranger-kms:/tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 13:05:40.716249 1522 mini_ranger_kms.cc:240] Using host: 127.1.124.148
I20260808 13:05:40.749318 1522 mini_ranger_kms.cc:276] Ranger KMS PID: 19645
I20260808 13:05:40.749532 1522 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.124.148:50545
Aug 08, 2026 1:05:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 1:05:41 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 1:05:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 1:05:41 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 1:05:42 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 1:05:42 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 13:05:43 dist-test-slave-t3q3 krb5kdc[18985](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786194343, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.124.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:05:43.636237 1522 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.733s user 0.137s sys 0.503s
I20260808 13:05:43.636632 1522 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 13:05:43.636915 1522 mini_ranger_kms.cc:401] 127.1.124.148:50545/kms/v1/keys
Aug 08 13:05:43 dist-test-slave-t3q3 krb5kdc[18985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194343, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.124.148@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
Aug 08 13:05:44 dist-test-slave-t3q3 krb5kdc[18985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194343, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.124.148@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
Aug 08 13:05:46 dist-test-slave-t3q3 krb5kdc[18985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:05:46.536223 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:42425
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:40735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:42425
--ranger_config_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.124.148:50545/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:05:46.991223 19726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:05:46.991519 19726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:05:46.991693 19726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:05:46.999007 19726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:05:46.999128 19726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:05:46.999238 19726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:05:46.999332 19726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:05:46.999415 19726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:05:47.010713 19726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:40735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.124.148:50545/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:42425
--ranger_config_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:42425
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:05:47.012193 19726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:05:47.013592 19726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:05:47.026001 19734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:05:47.026057 19732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:05:47.025952 19739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:05:47.026911 19726 server_base.cc:1034] running on GCE node
I20260808 13:05:47.027647 19726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:05:47.029090 19726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:05:47.030328 19726 hybrid_clock.cc:648] HybridClock initialized: now 1786194347030277 us; error 45 us; skew 500 ppm
I20260808 13:05:47.030732 19726 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:05:47 dist-test-slave-t3q3 krb5kdc[18985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194347, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:05:47.034231 19726 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:05:47.035856 19726 webserver.cc:460] Webserver started at http://127.1.124.190:35497/ using document root <none> and password file <none>
I20260808 13:05:47.036572 19726 fs_manager.cc:359] Metadata directory not provided
I20260808 13:05:47.036674 19726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:05:47.036948 19726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 13:05:47 dist-test-slave-t3q3 krb5kdc[18985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194347, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
I20260808 13:05:47.091634 19726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "5dccc735b47e473bb99dde6cfdf09a59"
format_stamp: "Formatted at 2026-08-08 13:05:47 on dist-test-slave-t3q3"
server_key: "bd67c328cfdf8ade6c695b0657e3c57d"
server_key_iv: "87a9270f71c1f31f41bd31ccee8a78b5"
server_key_version: "kuduclusterkey@0"
I20260808 13:05:47.092623 19726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "5dccc735b47e473bb99dde6cfdf09a59"
format_stamp: "Formatted at 2026-08-08 13:05:47 on dist-test-slave-t3q3"
server_key: "bd67c328cfdf8ade6c695b0657e3c57d"
server_key_iv: "87a9270f71c1f31f41bd31ccee8a78b5"
server_key_version: "kuduclusterkey@0"
I20260808 13:05:47.098642 19726 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.001s sys 0.004s
I20260808 13:05:47.116233 19748 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:05:47.117931 19726 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 13:05:47.118149 19726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "5dccc735b47e473bb99dde6cfdf09a59"
format_stamp: "Formatted at 2026-08-08 13:05:47 on dist-test-slave-t3q3"
server_key: "bd67c328cfdf8ade6c695b0657e3c57d"
server_key_iv: "87a9270f71c1f31f41bd31ccee8a78b5"
server_key_version: "kuduclusterkey@0"
I20260808 13:05:47.118422 19726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:05:47.132886 19726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:05:47.136941 19726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:05:47.137286 19726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:05:47.157058 19726 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:42425
I20260808 13:05:47.157114 19826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:42425 every 8 connection(s)
I20260808 13:05:47.158443 19726 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:05:47.162247 19827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:05:47.164161 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 19726
I20260808 13:05:47.164369 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:05:47.165262 1522 external_mini_cluster.cc:1428] Setting key 974de902e5f5a0f44643712c7dc9ef57
I20260808 13:05:47.175280 19827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59: Bootstrap starting.
I20260808 13:05:47.179517 19827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59: Neither blocks nor log segments found. Creating new log.
Aug 08 13:05:47 dist-test-slave-t3q3 krb5kdc[18985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:05:47.180768 19827 log.cc:824] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59: Log is configured to *not* fsync() on all Append() calls
I20260808 13:05:47.184580 19827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59: No bootstrap required, opened a new log
I20260808 13:05:47.190178 19834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:05:47.167450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51822 (local address 127.1.124.190:42425)
0808 13:05:47.168338 (+ 888us) server_negotiation.cc:206] Beginning negotiation
0808 13:05:47.168350 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:05:47.168398 (+ 48us) server_negotiation.cc:407] Connection header received
0808 13:05:47.170117 (+ 1719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:05:47.170142 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:05:47.170515 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:05:47.170844 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:05:47.172544 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:47.173902 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:47.175684 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:47.176444 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:47.176801 (+ 357us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:05:47.181584 (+ 4783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:05:47.181592 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:05:47.181608 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:05:47.181647 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:05:47.185440 (+ 3793us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:05:47.186175 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:05:47.186184 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:05:47.186195 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:05:47.186416 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:05:47.186807 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:05:47.186813 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:05:47.186816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:05:47.187566 (+ 750us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:05:47.187883 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0808 13:05:47.188285 (+ 402us) server_negotiation.cc:299] Negotiation successful
0808 13:05:47.188759 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":679,"thread_start_us":433,"threads_started":1}
I20260808 13:05:47.195019 19827 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dccc735b47e473bb99dde6cfdf09a59" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42425 } }
I20260808 13:05:47.195392 19827 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:05:47.195474 19827 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5dccc735b47e473bb99dde6cfdf09a59, State: Initialized, Role: FOLLOWER
I20260808 13:05:47.196192 19827 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dccc735b47e473bb99dde6cfdf09a59" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42425 } }
I20260808 13:05:47.196504 19827 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:05:47.196682 19827 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:05:47.196903 19827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:05:47.199224 19827 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dccc735b47e473bb99dde6cfdf09a59" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42425 } }
I20260808 13:05:47.199710 19827 leader_election.cc:302] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5dccc735b47e473bb99dde6cfdf09a59; no voters:
I20260808 13:05:47.200366 19827 leader_election.cc:288] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:05:47.200527 19836 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:05:47.202122 19836 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59 [term 1 LEADER]: Becoming Leader. State: Replica: 5dccc735b47e473bb99dde6cfdf09a59, State: Running, Role: LEADER
I20260808 13:05:47.202864 19836 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dccc735b47e473bb99dde6cfdf09a59" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42425 } }
I20260808 13:05:47.203161 19827 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:05:47.206156 19838 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5dccc735b47e473bb99dde6cfdf09a59. Latest consensus state: current_term: 1 leader_uuid: "5dccc735b47e473bb99dde6cfdf09a59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dccc735b47e473bb99dde6cfdf09a59" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42425 } } }
I20260808 13:05:47.206663 19838 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59 [sys.catalog]: This master's current role is: LEADER
I20260808 13:05:47.207177 19837 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5dccc735b47e473bb99dde6cfdf09a59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5dccc735b47e473bb99dde6cfdf09a59" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42425 } } }
I20260808 13:05:47.207573 19837 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59 [sys.catalog]: This master's current role is: LEADER
I20260808 13:05:47.208550 19851 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:05:47.215026 19851 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:05:47.216145 19827 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 13:05:47.244601 19851 catalog_manager.cc:1347] Generated new cluster ID: 2e7074166efc48de894bf16727c868e3
I20260808 13:05:47.244773 19851 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:05:47.306619 19851 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:05:47.308808 19851 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:05:47.322029 19851 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59: Generated new TSK 0
I20260808 13:05:47.323335 19851 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.124.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
I20260808 13:05:48.056030 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.124.148:50545/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42425
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:40735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 13:05:48 dist-test-slave-t3q3 krb5kdc[18985](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786194348, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 13:05:48.549628 19899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:05:48.550078 19899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:05:48.550242 19899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:05:48.558847 19899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:05:48.559021 19899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:05:48.559221 19899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:05:48.570012 19899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:40735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.124.148:50545/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:05:48.571590 19899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:05:48.573096 19899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:05:48.600728 19914 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:05:48.601385 19917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:05:48.614192 19921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:05:48.614430 19899 server_base.cc:1034] running on GCE node
I20260808 13:05:48.615196 19899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:05:48.618052 19899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:05:48.619395 19899 hybrid_clock.cc:648] HybridClock initialized: now 1786194348619330 us; error 57 us; skew 500 ppm
I20260808 13:05:48.619870 19899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:05:48 dist-test-slave-t3q3 krb5kdc[18985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194348, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:05:48.624727 19899 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:05:48.627204 19899 webserver.cc:460] Webserver started at http://127.1.124.129:39101/ using document root <none> and password file <none>
I20260808 13:05:48.628329 19899 fs_manager.cc:359] Metadata directory not provided
I20260808 13:05:48.628531 19899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:05:48.629046 19899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 13:05:48 dist-test-slave-t3q3 krb5kdc[18985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194348, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
I20260808 13:05:48.650382 19899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "f5d31980074448cf8804fa0838142017"
format_stamp: "Formatted at 2026-08-08 13:05:48 on dist-test-slave-t3q3"
server_key: "c6cc116d3d6cf0e2225562e3274884e0"
server_key_iv: "02465d7cae8b719cceab042ee13fc8a4"
server_key_version: "kuduclusterkey@0"
I20260808 13:05:48.652915 19899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "f5d31980074448cf8804fa0838142017"
format_stamp: "Formatted at 2026-08-08 13:05:48 on dist-test-slave-t3q3"
server_key: "c6cc116d3d6cf0e2225562e3274884e0"
server_key_iv: "02465d7cae8b719cceab042ee13fc8a4"
server_key_version: "kuduclusterkey@0"
I20260808 13:05:48.663087 19899 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.004s
I20260808 13:05:48.694748 19932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:05:48.697052 19899 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.009s sys 0.004s
I20260808 13:05:48.697364 19899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "f5d31980074448cf8804fa0838142017"
format_stamp: "Formatted at 2026-08-08 13:05:48 on dist-test-slave-t3q3"
server_key: "c6cc116d3d6cf0e2225562e3274884e0"
server_key_iv: "02465d7cae8b719cceab042ee13fc8a4"
server_key_version: "kuduclusterkey@0"
I20260808 13:05:48.697765 19899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:05:48.757931 19899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:05:48.763353 19899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:05:48.763797 19899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:05:48.771327 19899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:05:48.781826 19899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:05:48.782025 19899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260808 13:05:48.782248 19899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:05:48.782377 19899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 13:05:48 dist-test-slave-t3q3 krb5kdc[18985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194348, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
I20260808 13:05:49.238940 19899 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:45195
I20260808 13:05:49.238996 20109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:45195 every 8 connection(s)
I20260808 13:05:49.240895 19899 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:05:49.244627 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 19899
I20260808 13:05:49.244832 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:05:49.245651 1522 external_mini_cluster.cc:1428] Setting key ece63b471746dac8087f48c90d62aeca
Aug 08 13:05:49 dist-test-slave-t3q3 krb5kdc[18985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194348, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.124.130@KRBTEST.COM; defaulting to no policy
I20260808 13:05:49.286988 20114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:05:49.252104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:56337 (local address 127.1.124.190:42425)
0808 13:05:49.261287 (+ 9183us) server_negotiation.cc:206] Beginning negotiation
0808 13:05:49.261294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:05:49.262285 (+ 991us) server_negotiation.cc:407] Connection header received
0808 13:05:49.265104 (+ 2819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:05:49.265111 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:05:49.265197 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:05:49.265278 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:05:49.267586 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:49.268812 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:49.271241 (+ 2429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:49.271890 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:49.271992 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:05:49.276840 (+ 4848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:05:49.276847 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:05:49.276853 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:05:49.276891 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:05:49.281073 (+ 4182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:05:49.282069 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:05:49.282076 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:05:49.282079 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:05:49.282248 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:05:49.283492 (+ 1244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:05:49.283500 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:05:49.283505 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:05:49.283887 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:05:49.284024 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 13:05:49.285187 (+ 1163us) server_negotiation.cc:299] Negotiation successful
0808 13:05:49.285562 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9037,"thread_start_us":8908,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.130@KRBTEST.COM" created.
Aug 08 13:05:49 dist-test-slave-t3q3 krb5kdc[18985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194348, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
I20260808 13:05:49.289444 20115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:05:49.252813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:42425 (local address 127.1.124.129:56337)
0808 13:05:49.261849 (+ 9036us) negotiation.cc:107] Waiting for socket to connect
0808 13:05:49.261902 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 13:05:49.264750 (+ 2848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:05:49.265828 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:05:49.265839 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:05:49.266499 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:05:49.267369 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:05:49.267397 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:05:49.269041 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:05:49.269046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:05:49.271071 (+ 2025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:05:49.271087 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:05:49.272393 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:05:49.272401 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:05:49.272674 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:05:49.273539 (+ 865us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:05:49.273570 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:05:49.276571 (+ 3001us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:05:49.281252 (+ 4681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:05:49.281262 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:05:49.281280 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:05:49.281911 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:05:49.283051 (+ 1140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:05:49.283055 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:05:49.283059 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:05:49.283325 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:05:49.284176 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:05:49.284184 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:05:49.284677 (+ 493us) client_negotiation.cc:769] Sending connection context
0808 13:05:49.287335 (+ 2658us) client_negotiation.cc:240] Negotiation successful
0808 13:05:49.287928 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8711,"thread_start_us":10234,"threads_started":1}
I20260808 13:05:49.298454 20110 heartbeater.cc:347] Connected to a master server at 127.1.124.190:42425
I20260808 13:05:49.298838 20110 heartbeater.cc:464] Registering TS with master...
I20260808 13:05:49.299830 20110 heartbeater.cc:511] Master 127.1.124.190:42425 requested a full tablet report, sending...
I20260808 13:05:49.302450 19772 ts_manager.cc:194] Registered new tserver with Master: f5d31980074448cf8804fa0838142017 (127.1.124.129:45195)
I20260808 13:05:49.304867 19772 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:56337
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
I20260808 13:05:49.340494 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.124.148:50545/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42425
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:40735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:05:49.775329 20122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:05:49.775676 20122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:05:49.775844 20122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:05:49.783684 20122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:05:49.783847 20122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:05:49.784037 20122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:05:49.795166 20122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:40735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.124.148:50545/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:05:49.796661 20122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:05:49.798226 20122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:05:49.813742 20140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:05:49.814247 20142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:05:49.815099 20122 server_base.cc:1034] running on GCE node
W20260808 13:05:49.815104 20146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:05:49.815959 20122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:05:49.817207 20122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:05:49.818470 20122 hybrid_clock.cc:648] HybridClock initialized: now 1786194349818421 us; error 53 us; skew 500 ppm
I20260808 13:05:49.818876 20122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:05:49 dist-test-slave-t3q3 krb5kdc[18985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194349, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:05:49.822460 20122 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:05:49.824306 20122 webserver.cc:460] Webserver started at http://127.1.124.130:44383/ using document root <none> and password file <none>
I20260808 13:05:49.825071 20122 fs_manager.cc:359] Metadata directory not provided
I20260808 13:05:49.825246 20122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:05:49.825598 20122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 13:05:49 dist-test-slave-t3q3 krb5kdc[18985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194349, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
I20260808 13:05:49.844218 20122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "481c6027a95c40828cab372c5f63a2f0"
format_stamp: "Formatted at 2026-08-08 13:05:49 on dist-test-slave-t3q3"
server_key: "d7fb2591dddb51b54e33fc648b2e7852"
server_key_iv: "a96c03721b025c3f6862808b83ac60e7"
server_key_version: "kuduclusterkey@0"
I20260808 13:05:49.845445 20122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "481c6027a95c40828cab372c5f63a2f0"
format_stamp: "Formatted at 2026-08-08 13:05:49 on dist-test-slave-t3q3"
server_key: "d7fb2591dddb51b54e33fc648b2e7852"
server_key_iv: "a96c03721b025c3f6862808b83ac60e7"
server_key_version: "kuduclusterkey@0"
I20260808 13:05:49.853343 20122 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260808 13:05:49.871475 20157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:05:49.873397 20122 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260808 13:05:49.873594 20122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "481c6027a95c40828cab372c5f63a2f0"
format_stamp: "Formatted at 2026-08-08 13:05:49 on dist-test-slave-t3q3"
server_key: "d7fb2591dddb51b54e33fc648b2e7852"
server_key_iv: "a96c03721b025c3f6862808b83ac60e7"
server_key_version: "kuduclusterkey@0"
I20260808 13:05:49.873883 20122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:05:49.893092 20122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:05:49.898268 20122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:05:49.898623 20122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:05:49.899809 20122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:05:49.901647 20122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:05:49.901746 20122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:05:49.901872 20122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:05:49.901935 20122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:05:49.952514 20122 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:33595
I20260808 13:05:49.952611 20326 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:33595 every 8 connection(s)
I20260808 13:05:49.953855 20122 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:05:49.956158 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 20122
I20260808 13:05:49.956348 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:05:49.956805 1522 external_mini_cluster.cc:1428] Setting key fdd10fbbf7f17b9f6419d64ea1045278
Aug 08 13:05:49 dist-test-slave-t3q3 krb5kdc[18985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194349, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:05:49.981400 20329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:05:49.957602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:40403 (local address 127.1.124.190:42425)
0808 13:05:49.958139 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0808 13:05:49.958145 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:05:49.959200 (+ 1055us) server_negotiation.cc:407] Connection header received
0808 13:05:49.963021 (+ 3821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:05:49.963028 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:05:49.963110 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:05:49.963234 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:05:49.965339 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:49.966439 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:49.969030 (+ 2591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:49.969675 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:49.969800 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:05:49.973595 (+ 3795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:05:49.973605 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:05:49.973613 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:05:49.973660 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:05:49.977677 (+ 4017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:05:49.978493 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:05:49.978501 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:05:49.978508 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:05:49.978682 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:05:49.979189 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:05:49.979197 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:05:49.979203 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:05:49.979624 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:05:49.979775 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 13:05:49.980608 (+ 833us) server_negotiation.cc:299] Negotiation successful
0808 13:05:49.980890 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":297,"threads_started":1}
I20260808 13:05:49.982127 20330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:05:49.957731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:42425 (local address 127.1.124.130:40403)
0808 13:05:49.958879 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0808 13:05:49.958928 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 13:05:49.962710 (+ 3782us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:05:49.963498 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:05:49.963512 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:05:49.964228 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:05:49.965116 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:05:49.965145 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:05:49.966619 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:05:49.966627 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:05:49.968870 (+ 2243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:05:49.968888 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:05:49.969913 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:05:49.969922 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:05:49.970274 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:05:49.971114 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:05:49.971151 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:05:49.973331 (+ 2180us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:05:49.977822 (+ 4491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:05:49.977829 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:05:49.977843 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:05:49.978339 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:05:49.978829 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:05:49.978837 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:05:49.978841 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:05:49.979052 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:05:49.979759 (+ 707us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:05:49.979766 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:05:49.980192 (+ 426us) client_negotiation.cc:769] Sending connection context
0808 13:05:49.980435 (+ 243us) client_negotiation.cc:240] Negotiation successful
0808 13:05:49.980895 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"thread_start_us":644,"threads_started":1}
I20260808 13:05:49.983711 20327 heartbeater.cc:347] Connected to a master server at 127.1.124.190:42425
I20260808 13:05:49.984128 20327 heartbeater.cc:464] Registering TS with master...
I20260808 13:05:49.985069 20327 heartbeater.cc:511] Master 127.1.124.190:42425 requested a full tablet report, sending...
I20260808 13:05:49.986920 19772 ts_manager.cc:194] Registered new tserver with Master: 481c6027a95c40828cab372c5f63a2f0 (127.1.124.130:33595)
I20260808 13:05:49.987748 19772 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:40403
WARNING: no policy specified for kudu/127.1.124.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
I20260808 13:05:50.028579 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.124.148:50545/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42425
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:40735
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:05:50.308208 20110 heartbeater.cc:503] Master 127.1.124.190:42425 was elected leader, sending a full tablet report...
W20260808 13:05:50.392385 20333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:05:50.392724 20333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:05:50.392899 20333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:05:50.400079 20333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:05:50.400231 20333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:05:50.400410 20333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:05:50.411181 20333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:40735
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.124.148:50545/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42425
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:05:50.412586 20333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:05:50.413930 20333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:05:50.427884 20339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:05:50.427788 20342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:05:50.427834 20340 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:05:50.428966 20333 server_base.cc:1034] running on GCE node
I20260808 13:05:50.429756 20333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:05:50.430887 20333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:05:50.432178 20333 hybrid_clock.cc:648] HybridClock initialized: now 1786194350432098 us; error 62 us; skew 500 ppm
I20260808 13:05:50.432526 20333 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:05:50 dist-test-slave-t3q3 krb5kdc[18985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194350, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:05:50.435796 20333 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:05:50.437450 20333 webserver.cc:460] Webserver started at http://127.1.124.131:34107/ using document root <none> and password file <none>
I20260808 13:05:50.438141 20333 fs_manager.cc:359] Metadata directory not provided
I20260808 13:05:50.438301 20333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:05:50.438628 20333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 13:05:50 dist-test-slave-t3q3 krb5kdc[18985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194350, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
I20260808 13:05:50.455761 20333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "8947760f9927419f9241a45aba0701a6"
format_stamp: "Formatted at 2026-08-08 13:05:50 on dist-test-slave-t3q3"
server_key: "d085a166b4c22aa28530c3237cf37abc"
server_key_iv: "5c9441c39249a8f247f474f642ee54ac"
server_key_version: "kuduclusterkey@0"
I20260808 13:05:50.457024 20333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "8947760f9927419f9241a45aba0701a6"
format_stamp: "Formatted at 2026-08-08 13:05:50 on dist-test-slave-t3q3"
server_key: "d085a166b4c22aa28530c3237cf37abc"
server_key_iv: "5c9441c39249a8f247f474f642ee54ac"
server_key_version: "kuduclusterkey@0"
I20260808 13:05:50.464604 20333 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260808 13:05:50.482556 20350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:05:50.484246 20333 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 13:05:50.484447 20333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "8947760f9927419f9241a45aba0701a6"
format_stamp: "Formatted at 2026-08-08 13:05:50 on dist-test-slave-t3q3"
server_key: "d085a166b4c22aa28530c3237cf37abc"
server_key_iv: "5c9441c39249a8f247f474f642ee54ac"
server_key_version: "kuduclusterkey@0"
I20260808 13:05:50.484699 20333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:05:50.512777 20333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:05:50.516788 20333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:05:50.517124 20333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:05:50.518111 20333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:05:50.519630 20333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:05:50.519737 20333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:05:50.519896 20333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:05:50.519973 20333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:05:50.556362 20333 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:34533
I20260808 13:05:50.556461 20523 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:34533 every 8 connection(s)
I20260808 13:05:50.557821 20333 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:05:50.560478 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 20333
I20260808 13:05:50.560689 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:05:50.561298 1522 external_mini_cluster.cc:1428] Setting key faaf8b4c9ee80088af1ae90956d95096
Aug 08 13:05:50 dist-test-slave-t3q3 krb5kdc[18985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194350, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:05:50.578869 20528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:05:50.560657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:39373 (local address 127.1.124.190:42425)
0808 13:05:50.561221 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0808 13:05:50.561228 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:05:50.562312 (+ 1084us) server_negotiation.cc:407] Connection header received
0808 13:05:50.564351 (+ 2039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:05:50.564359 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:05:50.564448 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:05:50.564538 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:05:50.566056 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.566820 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.568455 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.568894 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.568992 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:05:50.572399 (+ 3407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:05:50.572404 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:05:50.572407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:05:50.572440 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:05:50.575327 (+ 2887us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:05:50.576447 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:05:50.576453 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:05:50.576456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:05:50.576635 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:05:50.577217 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:05:50.577222 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:05:50.577225 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:05:50.577571 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:05:50.577696 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 13:05:50.578423 (+ 727us) server_negotiation.cc:299] Negotiation successful
0808 13:05:50.578603 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":428,"thread_start_us":243,"threads_started":1}
I20260808 13:05:50.580019 20530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:05:50.561263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:42425 (local address 127.1.124.131:39373)
0808 13:05:50.562156 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0808 13:05:50.562205 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 13:05:50.564002 (+ 1797us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:05:50.564798 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:05:50.564807 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:05:50.565272 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:05:50.565874 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:05:50.565892 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.566955 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.566959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:05:50.568320 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:05:50.568333 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.569089 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.569093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:05:50.569328 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:05:50.569979 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:05:50.570006 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:05:50.572182 (+ 2176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:05:50.575477 (+ 3295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:05:50.575487 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:05:50.575505 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:05:50.576281 (+ 776us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:05:50.576754 (+ 473us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:05:50.576759 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:05:50.576762 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:05:50.577031 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:05:50.577694 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:05:50.577700 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:05:50.578082 (+ 382us) client_negotiation.cc:769] Sending connection context
0808 13:05:50.578326 (+ 244us) client_negotiation.cc:240] Negotiation successful
0808 13:05:50.578752 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":408,"threads_started":1}
I20260808 13:05:50.581609 20526 heartbeater.cc:347] Connected to a master server at 127.1.124.190:42425
I20260808 13:05:50.581970 20526 heartbeater.cc:464] Registering TS with master...
I20260808 13:05:50.582902 20526 heartbeater.cc:511] Master 127.1.124.190:42425 requested a full tablet report, sending...
I20260808 13:05:50.584528 19772 ts_manager.cc:194] Registered new tserver with Master: 8947760f9927419f9241a45aba0701a6 (127.1.124.131:34533)
I20260808 13:05:50.585376 19772 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:39373
I20260808 13:05:50.590265 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:05:50.609123 20528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:05:50.595146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42418 (local address 127.1.124.190:42425)
0808 13:05:50.595329 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 13:05:50.595333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:05:50.595916 (+ 583us) server_negotiation.cc:407] Connection header received
0808 13:05:50.596183 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:05:50.596188 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:05:50.596271 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:05:50.596360 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:05:50.597810 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.598605 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.599972 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.600472 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.600571 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:05:50.603324 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:05:50.603329 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:05:50.603332 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:05:50.603367 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:05:50.606268 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:05:50.607041 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:05:50.607045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:05:50.607047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:05:50.607186 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:05:50.607643 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:05:50.607646 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:05:50.607647 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:05:50.607928 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:05:50.608183 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0808 13:05:50.608637 (+ 454us) server_negotiation.cc:299] Negotiation successful
0808 13:05:50.608819 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 13:05:50.613431 19772 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42418:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:05:50.621554 19772 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:05:50.656888 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:05:50.638373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39796 (local address 127.1.124.130:33595)
0808 13:05:50.648541 (+ 10168us) server_negotiation.cc:206] Beginning negotiation
0808 13:05:50.648555 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 13:05:50.648596 (+ 41us) server_negotiation.cc:407] Connection header received
0808 13:05:50.648724 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:05:50.648737 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:05:50.649086 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:05:50.649329 (+ 243us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:05:50.650380 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.651815 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.654062 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.655602 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.655754 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:05:50.655980 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 13:05:50.656368 (+ 388us) server_negotiation.cc:299] Negotiation successful
0808 13:05:50.656537 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9999,"thread_start_us":9888,"threads_started":1}
I20260808 13:05:50.656912 20542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:05:50.637759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:33595 (local address 127.0.0.1:39796)
0808 13:05:50.646975 (+ 9216us) negotiation.cc:107] Waiting for socket to connect
0808 13:05:50.646998 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:05:50.647125 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:05:50.649654 (+ 2529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:05:50.649667 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:05:50.649704 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:05:50.650206 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:05:50.650217 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.651962 (+ 1745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.651970 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:05:50.653854 (+ 1884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:05:50.653869 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.655869 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.655875 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:05:50.656111 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:05:50.656164 (+ 53us) client_negotiation.cc:769] Sending connection context
0808 13:05:50.656307 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 13:05:50.656513 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9080,"thread_start_us":7263,"threads_started":1}
I20260808 13:05:50.658215 20546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:05:50.635365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46594 (local address 127.1.124.131:34533)
0808 13:05:50.648718 (+ 13353us) server_negotiation.cc:206] Beginning negotiation
0808 13:05:50.648724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:05:50.648752 (+ 28us) server_negotiation.cc:407] Connection header received
0808 13:05:50.648831 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:05:50.648836 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:05:50.649028 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:05:50.649294 (+ 266us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:05:50.650587 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.652362 (+ 1775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.655390 (+ 3028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.656772 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.656906 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:05:50.657136 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 13:05:50.657690 (+ 554us) server_negotiation.cc:299] Negotiation successful
0808 13:05:50.657852 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13201,"thread_start_us":14188,"threads_started":1}
I20260808 13:05:50.658669 20544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:05:50.635333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:45195 (local address 127.0.0.1:34694)
0808 13:05:50.646614 (+ 11281us) negotiation.cc:107] Waiting for socket to connect
0808 13:05:50.646642 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 13:05:50.646811 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:05:50.650881 (+ 4070us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:05:50.650885 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:05:50.650914 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:05:50.651331 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:05:50.651345 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.653657 (+ 2312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.653666 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:05:50.655876 (+ 2210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:05:50.655891 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.657844 (+ 1953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.657851 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:05:50.658012 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:05:50.658047 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 13:05:50.658122 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 13:05:50.658257 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11133,"mutex_wait_us":1,"thread_start_us":10986,"threads_started":1}
I20260808 13:05:50.658195 20551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:05:50.635012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:34533 (local address 127.0.0.1:46594)
0808 13:05:50.645490 (+ 10478us) negotiation.cc:107] Waiting for socket to connect
0808 13:05:50.645540 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 13:05:50.645766 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:05:50.649320 (+ 3554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:05:50.649327 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:05:50.649373 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:05:50.649866 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:05:50.649885 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.652545 (+ 2660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.652551 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:05:50.655218 (+ 2667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:05:50.655230 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.657335 (+ 2105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.657343 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:05:50.657493 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:05:50.657516 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 13:05:50.657627 (+ 111us) client_negotiation.cc:240] Negotiation successful
0808 13:05:50.657769 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10199,"mutex_wait_us":270,"thread_start_us":11655,"threads_started":1}
I20260808 13:05:50.659876 20549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:05:50.635555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34694 (local address 127.1.124.129:45195)
0808 13:05:50.650254 (+ 14699us) server_negotiation.cc:206] Beginning negotiation
0808 13:05:50.650263 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:05:50.650290 (+ 27us) server_negotiation.cc:407] Connection header received
0808 13:05:50.650389 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:05:50.650395 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:05:50.650631 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:05:50.651548 (+ 917us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:05:50.651793 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.653453 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.656129 (+ 2676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.657585 (+ 1456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.659038 (+ 1453us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:05:50.659267 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 13:05:50.659340 (+ 73us) server_negotiation.cc:299] Negotiation successful
0808 13:05:50.659507 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14502,"thread_start_us":14310,"threads_started":1}
I20260808 13:05:50.661644 20232 tablet_service.cc:1467] Processing CreateTablet for tablet 9ebd7dfa07ad4bcdbc43628c397a43a5 (DEFAULT_TABLE table=test-table [id=edab49b57ab64a0097f779e363c1bfdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:05:50.663285 20232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ebd7dfa07ad4bcdbc43628c397a43a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:05:50.662915 20011 tablet_service.cc:1467] Processing CreateTablet for tablet 9ebd7dfa07ad4bcdbc43628c397a43a5 (DEFAULT_TABLE table=test-table [id=edab49b57ab64a0097f779e363c1bfdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:05:50.663465 20422 tablet_service.cc:1467] Processing CreateTablet for tablet 9ebd7dfa07ad4bcdbc43628c397a43a5 (DEFAULT_TABLE table=test-table [id=edab49b57ab64a0097f779e363c1bfdc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:05:50.664748 20011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ebd7dfa07ad4bcdbc43628c397a43a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:05:50.664913 20422 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ebd7dfa07ad4bcdbc43628c397a43a5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:05:50.676630 20553 tablet_bootstrap.cc:492] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017: Bootstrap starting.
I20260808 13:05:50.678680 20556 tablet_bootstrap.cc:492] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 481c6027a95c40828cab372c5f63a2f0: Bootstrap starting.
I20260808 13:05:50.680380 20557 tablet_bootstrap.cc:492] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 8947760f9927419f9241a45aba0701a6: Bootstrap starting.
I20260808 13:05:50.682420 20553 tablet_bootstrap.cc:654] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017: Neither blocks nor log segments found. Creating new log.
I20260808 13:05:50.682561 20556 tablet_bootstrap.cc:654] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 481c6027a95c40828cab372c5f63a2f0: Neither blocks nor log segments found. Creating new log.
I20260808 13:05:50.683614 20557 tablet_bootstrap.cc:654] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 8947760f9927419f9241a45aba0701a6: Neither blocks nor log segments found. Creating new log.
I20260808 13:05:50.683895 20556 log.cc:824] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 481c6027a95c40828cab372c5f63a2f0: Log is configured to *not* fsync() on all Append() calls
I20260808 13:05:50.683895 20553 log.cc:824] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017: Log is configured to *not* fsync() on all Append() calls
I20260808 13:05:50.684808 20557 log.cc:824] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 8947760f9927419f9241a45aba0701a6: Log is configured to *not* fsync() on all Append() calls
I20260808 13:05:50.687397 20553 tablet_bootstrap.cc:492] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017: No bootstrap required, opened a new log
I20260808 13:05:50.687687 20553 ts_tablet_manager.cc:1397] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.002s
I20260808 13:05:50.687924 20556 tablet_bootstrap.cc:492] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 481c6027a95c40828cab372c5f63a2f0: No bootstrap required, opened a new log
I20260808 13:05:50.687922 20557 tablet_bootstrap.cc:492] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 8947760f9927419f9241a45aba0701a6: No bootstrap required, opened a new log
I20260808 13:05:50.688217 20556 ts_tablet_manager.cc:1397] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 481c6027a95c40828cab372c5f63a2f0: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260808 13:05:50.688241 20557 ts_tablet_manager.cc:1397] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 8947760f9927419f9241a45aba0701a6: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.002s
I20260808 13:05:50.696200 20553 raft_consensus.cc:348] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8947760f9927419f9241a45aba0701a6" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34533 } } peers { permanent_uuid: "f5d31980074448cf8804fa0838142017" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45195 } } peers { permanent_uuid: "481c6027a95c40828cab372c5f63a2f0" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33595 } }
I20260808 13:05:50.696652 20553 raft_consensus.cc:374] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:05:50.696789 20553 raft_consensus.cc:729] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5d31980074448cf8804fa0838142017, State: Initialized, Role: FOLLOWER
I20260808 13:05:50.697607 20553 consensus_queue.cc:260] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8947760f9927419f9241a45aba0701a6" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34533 } } peers { permanent_uuid: "f5d31980074448cf8804fa0838142017" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45195 } } peers { permanent_uuid: "481c6027a95c40828cab372c5f63a2f0" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33595 } }
I20260808 13:05:50.699585 20557 raft_consensus.cc:348] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 8947760f9927419f9241a45aba0701a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8947760f9927419f9241a45aba0701a6" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34533 } } peers { permanent_uuid: "f5d31980074448cf8804fa0838142017" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45195 } } peers { permanent_uuid: "481c6027a95c40828cab372c5f63a2f0" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33595 } }
I20260808 13:05:50.700119 20553 ts_tablet_manager.cc:1428] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017: Time spent starting tablet: real 0.012s user 0.012s sys 0.001s
I20260808 13:05:50.700243 20557 raft_consensus.cc:374] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 8947760f9927419f9241a45aba0701a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:05:50.700423 20557 raft_consensus.cc:729] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 8947760f9927419f9241a45aba0701a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8947760f9927419f9241a45aba0701a6, State: Initialized, Role: FOLLOWER
I20260808 13:05:50.701220 20557 consensus_queue.cc:260] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 8947760f9927419f9241a45aba0701a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8947760f9927419f9241a45aba0701a6" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34533 } } peers { permanent_uuid: "f5d31980074448cf8804fa0838142017" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45195 } } peers { permanent_uuid: "481c6027a95c40828cab372c5f63a2f0" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33595 } }
I20260808 13:05:50.701908 20556 raft_consensus.cc:348] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 481c6027a95c40828cab372c5f63a2f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8947760f9927419f9241a45aba0701a6" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34533 } } peers { permanent_uuid: "f5d31980074448cf8804fa0838142017" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45195 } } peers { permanent_uuid: "481c6027a95c40828cab372c5f63a2f0" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33595 } }
I20260808 13:05:50.702461 20556 raft_consensus.cc:374] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 481c6027a95c40828cab372c5f63a2f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:05:50.702607 20556 raft_consensus.cc:729] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 481c6027a95c40828cab372c5f63a2f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 481c6027a95c40828cab372c5f63a2f0, State: Initialized, Role: FOLLOWER
I20260808 13:05:50.702965 20526 heartbeater.cc:503] Master 127.1.124.190:42425 was elected leader, sending a full tablet report...
I20260808 13:05:50.703361 20556 consensus_queue.cc:260] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 481c6027a95c40828cab372c5f63a2f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8947760f9927419f9241a45aba0701a6" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34533 } } peers { permanent_uuid: "f5d31980074448cf8804fa0838142017" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45195 } } peers { permanent_uuid: "481c6027a95c40828cab372c5f63a2f0" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33595 } }
I20260808 13:05:50.703789 20557 ts_tablet_manager.cc:1428] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 8947760f9927419f9241a45aba0701a6: Time spent starting tablet: real 0.015s user 0.013s sys 0.002s
I20260808 13:05:50.705054 20327 heartbeater.cc:503] Master 127.1.124.190:42425 was elected leader, sending a full tablet report...
I20260808 13:05:50.707226 20556 ts_tablet_manager.cc:1428] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 481c6027a95c40828cab372c5f63a2f0: Time spent starting tablet: real 0.019s user 0.013s sys 0.001s
I20260808 13:05:50.749388 20562 raft_consensus.cc:482] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:05:50.749763 20562 raft_consensus.cc:504] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8947760f9927419f9241a45aba0701a6" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34533 } } peers { permanent_uuid: "f5d31980074448cf8804fa0838142017" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45195 } } peers { permanent_uuid: "481c6027a95c40828cab372c5f63a2f0" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33595 } }
I20260808 13:05:50.751636 20562 leader_election.cc:288] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8947760f9927419f9241a45aba0701a6 (127.1.124.131:34533), 481c6027a95c40828cab372c5f63a2f0 (127.1.124.130:33595)
I20260808 13:05:50.758597 20546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:05:50.752003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:38135 (local address 127.1.124.131:34533)
0808 13:05:50.752307 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0808 13:05:50.752313 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:05:50.753020 (+ 707us) server_negotiation.cc:407] Connection header received
0808 13:05:50.753282 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:05:50.753286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:05:50.753348 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:05:50.753451 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:05:50.754047 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.754775 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.756811 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.757610 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.757819 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:05:50.757901 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 13:05:50.758209 (+ 308us) server_negotiation.cc:299] Negotiation successful
0808 13:05:50.758340 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260808 13:05:50.758687 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:05:50.752323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:60617 (local address 127.1.124.130:33595)
0808 13:05:50.752542 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 13:05:50.752550 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:05:50.753269 (+ 719us) server_negotiation.cc:407] Connection header received
0808 13:05:50.753469 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:05:50.753476 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:05:50.753554 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:05:50.753660 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:05:50.754302 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.755028 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.756887 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.757711 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.757903 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:05:50.757985 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 13:05:50.758278 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 13:05:50.758407 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 13:05:50.758764 20572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:05:50.752000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:34533 (local address 127.1.124.129:38135)
0808 13:05:50.752874 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0808 13:05:50.752891 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:05:50.753113 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:05:50.753469 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:05:50.753474 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:05:50.753498 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:05:50.753885 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:05:50.753897 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.754908 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.754916 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:05:50.756637 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:05:50.756650 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.757832 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.757838 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:05:50.758024 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:05:50.758049 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 13:05:50.758155 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 13:05:50.758289 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"mutex_wait_us":21,"thread_start_us":454,"threads_started":1}
I20260808 13:05:50.758816 20573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:05:50.752002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:33595 (local address 127.1.124.129:60617)
0808 13:05:50.753154 (+ 1152us) negotiation.cc:107] Waiting for socket to connect
0808 13:05:50.753169 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:05:50.753333 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:05:50.753678 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:05:50.753682 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:05:50.753711 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:05:50.754147 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:05:50.754156 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.755177 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.755182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:05:50.756717 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:05:50.756729 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.757956 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.757963 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:05:50.758113 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:05:50.758137 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 13:05:50.758241 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 13:05:50.758367 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1026,"mutex_wait_us":426,"thread_start_us":400,"threads_started":1}
I20260808 13:05:50.760141 20260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebd7dfa07ad4bcdbc43628c397a43a5" candidate_uuid: "f5d31980074448cf8804fa0838142017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "481c6027a95c40828cab372c5f63a2f0" is_pre_election: true
I20260808 13:05:50.760156 20452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebd7dfa07ad4bcdbc43628c397a43a5" candidate_uuid: "f5d31980074448cf8804fa0838142017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8947760f9927419f9241a45aba0701a6" is_pre_election: true
I20260808 13:05:50.760670 20452 raft_consensus.cc:2454] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 8947760f9927419f9241a45aba0701a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5d31980074448cf8804fa0838142017 in term 0.
I20260808 13:05:50.760669 20260 raft_consensus.cc:2454] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 481c6027a95c40828cab372c5f63a2f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f5d31980074448cf8804fa0838142017 in term 0.
I20260808 13:05:50.761353 19933 leader_election.cc:302] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 481c6027a95c40828cab372c5f63a2f0, 8947760f9927419f9241a45aba0701a6, f5d31980074448cf8804fa0838142017; no voters:
I20260808 13:05:50.761785 20562 raft_consensus.cc:2781] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:05:50.761937 20562 raft_consensus.cc:482] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:05:50.762064 20562 raft_consensus.cc:3037] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017 [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:05:50.762439 20112 tablet.cc:2369] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:05:50.764537 20562 raft_consensus.cc:504] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8947760f9927419f9241a45aba0701a6" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34533 } } peers { permanent_uuid: "f5d31980074448cf8804fa0838142017" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45195 } } peers { permanent_uuid: "481c6027a95c40828cab372c5f63a2f0" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33595 } }
I20260808 13:05:50.765211 20562 leader_election.cc:288] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017 [CANDIDATE]: Term 1 election: Requested vote from peers 8947760f9927419f9241a45aba0701a6 (127.1.124.131:34533), 481c6027a95c40828cab372c5f63a2f0 (127.1.124.130:33595)
I20260808 13:05:50.765728 20452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebd7dfa07ad4bcdbc43628c397a43a5" candidate_uuid: "f5d31980074448cf8804fa0838142017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8947760f9927419f9241a45aba0701a6"
I20260808 13:05:50.765759 20260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ebd7dfa07ad4bcdbc43628c397a43a5" candidate_uuid: "f5d31980074448cf8804fa0838142017" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "481c6027a95c40828cab372c5f63a2f0"
I20260808 13:05:50.765965 20260 raft_consensus.cc:3037] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 481c6027a95c40828cab372c5f63a2f0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:05:50.765965 20452 raft_consensus.cc:3037] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 8947760f9927419f9241a45aba0701a6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:05:50.769435 20260 raft_consensus.cc:2454] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 481c6027a95c40828cab372c5f63a2f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5d31980074448cf8804fa0838142017 in term 1.
I20260808 13:05:50.769439 20452 raft_consensus.cc:2454] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 8947760f9927419f9241a45aba0701a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f5d31980074448cf8804fa0838142017 in term 1.
I20260808 13:05:50.769932 19933 leader_election.cc:302] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8947760f9927419f9241a45aba0701a6, f5d31980074448cf8804fa0838142017; no voters:
I20260808 13:05:50.770272 20562 raft_consensus.cc:2781] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:05:50.770841 20562 raft_consensus.cc:686] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017 [term 1 LEADER]: Becoming Leader. State: Replica: f5d31980074448cf8804fa0838142017, State: Running, Role: LEADER
I20260808 13:05:50.771404 20562 consensus_queue.cc:237] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8947760f9927419f9241a45aba0701a6" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34533 } } peers { permanent_uuid: "f5d31980074448cf8804fa0838142017" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45195 } } peers { permanent_uuid: "481c6027a95c40828cab372c5f63a2f0" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33595 } }
I20260808 13:05:50.775861 19766 catalog_manager.cc:5495] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017 reported cstate change: term changed from 0 to 1, leader changed from <none> to f5d31980074448cf8804fa0838142017 (127.1.124.129). New cstate: current_term: 1 leader_uuid: "f5d31980074448cf8804fa0838142017" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8947760f9927419f9241a45aba0701a6" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34533 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5d31980074448cf8804fa0838142017" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "481c6027a95c40828cab372c5f63a2f0" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33595 } health_report { overall_health: UNKNOWN } } }
W20260808 13:05:50.809653 20527 tablet.cc:2369] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 8947760f9927419f9241a45aba0701a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:05:50.815865 20549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:05:50.810094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34708 (local address 127.1.124.129:45195)
0808 13:05:50.810325 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 13:05:50.810329 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:05:50.810411 (+ 82us) server_negotiation.cc:407] Connection header received
0808 13:05:50.810688 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:05:50.810693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:05:50.810788 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:05:50.810936 (+ 148us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:05:50.811403 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.812136 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.813686 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.814088 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.814188 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:05:50.814689 (+ 501us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:05:50.815231 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:05:50.815341 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 13:05:50.815482 (+ 141us) server_negotiation.cc:299] Negotiation successful
0808 13:05:50.815554 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 13:05:50.824512 20452 raft_consensus.cc:1261] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 8947760f9927419f9241a45aba0701a6 [term 1 FOLLOWER]: Refusing update from remote peer f5d31980074448cf8804fa0838142017: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:05:50.824591 20260 raft_consensus.cc:1261] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 481c6027a95c40828cab372c5f63a2f0 [term 1 FOLLOWER]: Refusing update from remote peer f5d31980074448cf8804fa0838142017: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:05:50.825637 20578 consensus_queue.cc:1035] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8947760f9927419f9241a45aba0701a6" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:05:50.826004 20562 consensus_queue.cc:1035] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017 [LEADER]: Connected to new peer: Peer: permanent_uuid: "481c6027a95c40828cab372c5f63a2f0" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:05:50.841284 20585 mvcc.cc:205] Tried to move back new op lower bound from 7316252060966486016 to 7316252060763836416. Current Snapshot: MvccSnapshot[applied={T|T < 7316252060966486016}]
I20260808 13:05:50.852316 19766 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:05:50.852643 19766 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42418:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:05:50.856581 19766 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5dccc735b47e473bb99dde6cfdf09a59: Sending DeleteTablet for 3 replicas of tablet 9ebd7dfa07ad4bcdbc43628c397a43a5
I20260808 13:05:50.857847 20422 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ebd7dfa07ad4bcdbc43628c397a43a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:05:50 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:46594
I20260808 13:05:50.857963 20232 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ebd7dfa07ad4bcdbc43628c397a43a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:05:50 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:39796
I20260808 13:05:50.857969 20011 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ebd7dfa07ad4bcdbc43628c397a43a5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:05:50 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:34694
I20260808 13:05:50.861431 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 19899
I20260808 13:05:50.876967 20596 tablet_replica.cc:330] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 481c6027a95c40828cab372c5f63a2f0: stopping tablet replica
I20260808 13:05:50.877907 20596 raft_consensus.cc:2229] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 481c6027a95c40828cab372c5f63a2f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:05:50.878804 20596 raft_consensus.cc:2258] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 481c6027a95c40828cab372c5f63a2f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:05:50.880505 20598 tablet_replica.cc:330] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017: stopping tablet replica
I20260808 13:05:50.880810 20594 tablet_replica.cc:330] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 8947760f9927419f9241a45aba0701a6: stopping tablet replica
I20260808 13:05:50.880968 20598 raft_consensus.cc:2229] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:05:50.881273 20594 raft_consensus.cc:2229] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 8947760f9927419f9241a45aba0701a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:05:50.881724 20596 ts_tablet_manager.cc:1905] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 481c6027a95c40828cab372c5f63a2f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:05:50.881742 20598 raft_consensus.cc:2258] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:05:50.881821 20594 raft_consensus.cc:2258] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 8947760f9927419f9241a45aba0701a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:05:50.884784 20594 ts_tablet_manager.cc:1905] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 8947760f9927419f9241a45aba0701a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:05:50.884794 20598 ts_tablet_manager.cc:1905] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:05:50.892597 20594 ts_tablet_manager.cc:1918] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 8947760f9927419f9241a45aba0701a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 08 13:05:50 dist-test-slave-t3q3 krb5kdc[18985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.129@KRBTEST.COM
I20260808 13:05:50.892783 20594 log.cc:1196] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 8947760f9927419f9241a45aba0701a6: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/9ebd7dfa07ad4bcdbc43628c397a43a5
I20260808 13:05:50.893414 20594 ts_tablet_manager.cc:1939] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 8947760f9927419f9241a45aba0701a6: Deleting consensus metadata
I20260808 13:05:50.896435 19749 catalog_manager.cc:4841] TS 8947760f9927419f9241a45aba0701a6 (127.1.124.131:34533): tablet 9ebd7dfa07ad4bcdbc43628c397a43a5 (table test-table [id=edab49b57ab64a0097f779e363c1bfdc]) successfully deleted
I20260808 13:05:50.900030 20596 ts_tablet_manager.cc:1918] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 481c6027a95c40828cab372c5f63a2f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:05:50.900389 20596 log.cc:1196] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 481c6027a95c40828cab372c5f63a2f0: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/9ebd7dfa07ad4bcdbc43628c397a43a5
I20260808 13:05:50.901052 20596 ts_tablet_manager.cc:1939] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P 481c6027a95c40828cab372c5f63a2f0: Deleting consensus metadata
I20260808 13:05:50.900794 20598 ts_tablet_manager.cc:1918] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:05:50.901835 20598 log.cc:1196] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/9ebd7dfa07ad4bcdbc43628c397a43a5
I20260808 13:05:50.902455 20598 ts_tablet_manager.cc:1939] T 9ebd7dfa07ad4bcdbc43628c397a43a5 P f5d31980074448cf8804fa0838142017: Deleting consensus metadata
I20260808 13:05:50.902913 20549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:05:50.863074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34722 (local address 127.1.124.129:45195)
0808 13:05:50.863272 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 13:05:50.863279 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:05:50.880271 (+ 16992us) server_negotiation.cc:407] Connection header received
0808 13:05:50.880595 (+ 324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:05:50.880603 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:05:50.880720 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:05:50.880813 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:05:50.883507 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.884786 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.886909 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:50.887613 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:50.887726 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:05:50.894436 (+ 6710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:05:50.894452 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:05:50.894479 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:05:50.894539 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:05:50.898792 (+ 4253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:05:50.899866 (+ 1074us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:05:50.899876 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:05:50.899879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:05:50.900110 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:05:50.900735 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:05:50.900742 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:05:50.900745 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:05:50.901322 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:05:50.901630 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0808 13:05:50.902203 (+ 573us) server_negotiation.cc:299] Negotiation successful
0808 13:05:50.902480 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 13:05:50.903312 19752 catalog_manager.cc:4841] TS 481c6027a95c40828cab372c5f63a2f0 (127.1.124.130:33595): tablet 9ebd7dfa07ad4bcdbc43628c397a43a5 (table test-table [id=edab49b57ab64a0097f779e363c1bfdc]) successfully deleted
I20260808 13:05:50.904392 19750 catalog_manager.cc:4841] TS f5d31980074448cf8804fa0838142017 (127.1.124.129:45195): tablet 9ebd7dfa07ad4bcdbc43628c397a43a5 (table test-table [id=edab49b57ab64a0097f779e363c1bfdc]) successfully deleted
I20260808 13:05:50.905408 20098 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:05:51.008735 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 19899
I20260808 13:05:51.037194 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 20122
Aug 08 13:05:51 dist-test-slave-t3q3 krb5kdc[18985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.130@KRBTEST.COM
I20260808 13:05:51.053295 20552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:05:51.038190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39808 (local address 127.1.124.130:33595)
0808 13:05:51.038392 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 13:05:51.038397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:05:51.038416 (+ 19us) server_negotiation.cc:407] Connection header received
0808 13:05:51.038578 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:05:51.038582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:05:51.038659 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:05:51.038745 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:05:51.040052 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:51.040884 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:51.042305 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:51.042701 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:51.042818 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:05:51.047283 (+ 4465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:05:51.047301 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:05:51.047318 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:05:51.047371 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:05:51.050491 (+ 3120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:05:51.051175 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:05:51.051183 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:05:51.051185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:05:51.051320 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:05:51.051690 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:05:51.051695 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:05:51.051697 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:05:51.052264 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:05:51.052512 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0808 13:05:51.052804 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 13:05:51.053031 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 13:05:51.054008 20321 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:05:51.146780 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 20122
I20260808 13:05:51.175016 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 20333
Aug 08 13:05:51 dist-test-slave-t3q3 krb5kdc[18985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194346, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.131@KRBTEST.COM
I20260808 13:05:51.191525 20546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:05:51.175907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46600 (local address 127.1.124.131:34533)
0808 13:05:51.176188 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0808 13:05:51.176193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:05:51.176214 (+ 21us) server_negotiation.cc:407] Connection header received
0808 13:05:51.176335 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:05:51.176339 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:05:51.176416 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:05:51.176505 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:05:51.177938 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:51.178711 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:51.179984 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:05:51.180597 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:05:51.180700 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:05:51.185450 (+ 4750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:05:51.185468 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:05:51.185485 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:05:51.185521 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:05:51.188716 (+ 3195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:05:51.189554 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:05:51.189562 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:05:51.189565 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:05:51.189691 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:05:51.190088 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:05:51.190091 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:05:51.190093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:05:51.190559 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:05:51.190778 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0808 13:05:51.191085 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 13:05:51.191256 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260808 13:05:51.192476 20518 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:05:51.281812 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 20333
I20260808 13:05:51.304999 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 19726
I20260808 13:05:51.306289 19816 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:05:51.423171 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 19726
I20260808 13:05:51.454653 1522 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 13:05:51.800549 1522 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 13:05:51.800897 1522 mini_ranger.cc:66] Stopping Ranger...
I20260808 13:05:52.146993 1522 mini_ranger.cc:68] Stopped Ranger
2026-08-08 13:05:52.147 UTC [19037] LOG: received smart shutdown request
2026-08-08 13:05:52.151 UTC [19037] LOG: background worker "logical replication launcher" (PID 19049) exited with exit code 1
2026-08-08 13:05:52.152 UTC [19042] LOG: shutting down
2026-08-08 13:05:52.327 UTC [19037] LOG: database system is shut down
2026-08-08T13:05:52Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (54741 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:05:52 dist-test-slave-t3q3 krb5kdc[20619](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:05:52 dist-test-slave-t3q3 krb5kdc[20619](info): set up 2 sockets
Aug 08 13:05:52 dist-test-slave-t3q3 krb5kdc[20619](info): commencing operation
krb5kdc: starting...
W20260808 13:05:53.849078 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.343s user 0.003s sys 0.036s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:05:54 dist-test-slave-t3q3 krb5kdc[20619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:05:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:05:54Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.1.124.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.1.124.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.1.124.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/rangeradmin_127.1.124.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.1.124.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/rangeradmin_127.1.124.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.1.124.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.1.124.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.1.124.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/rangerlookup_127.1.124.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.1.124.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/rangerlookup_127.1.124.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.124.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.1.124.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.124.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/HTTP_127.1.124.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.124.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/HTTP_127.1.124.148@KRBTEST.COM.keytab.
I20260808 13:05:54.277223 1522 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-task9J1ueI/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/postgres -l logfile start
2026-08-08 13:05:56.890 UTC [20667] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 13:05:56.890 UTC [20667] LOG: listening on IPv4 address "127.1.124.148", port 54297
2026-08-08 13:05:56.895 UTC [20667] LOG: listening on Unix socket "/tmp/.s.PGSQL.54297"
2026-08-08 13:05:56.920 UTC [20672] LOG: database system was shut down at 2026-08-08 13:05:55 UTC
2026-08-08 13:05:56.924 UTC [20667] LOG: database system is ready to accept connections
I20260808 13:05:58.111127 1522 mini_postgres.cc:96] Postgres bound to 54297
2026-08-08 13:05:58.135 UTC [20681] FATAL: database "slave" does not exist
127.1.124.148:54297 - accepting connections
I20260808 13:05:58.136157 1522 mini_ranger.cc:161] Starting Ranger...
I20260808 13:05:58.169471 1522 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 13:05:58.695927 1522 mini_ranger.cc:87] Created ranger Postgres database
I20260808 13:05:58.696082 1522 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ranger-admin
2026-08-08 13:05:59,015 [I] DB FLAVOR :POSTGRES
2026-08-08 13:05:59,015 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 13:05:59,015 [I] Checking connection..
2026-08-08 13:05:59,016 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 13:05:59,300 [I] Checking connection passed.
2026-08-08 13:05:59,300 [I] --------- Verifying version history table ---------
2026-08-08 13:05:59,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 13:05:59,569 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 13:05:59,570 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 13:05:59,816 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 13:05:59,816 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 13:05:59,816 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 13:06:00.078 UTC [20783] WARNING: there is no transaction in progress
2026-08-08 13:06:00,085 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 13:06:00,086 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 13:06:00,330 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 13:06:00,330 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 13:06:00,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 13:06:00,601 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 13:06:00,862 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-t3q3.c.gcp-upstream.internal','N') ;"
2026-08-08 13:06:01,107 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 13:06:01,363 [I] Table x_portal_user does not exist in database ranger
2026-08-08 13:06:01,364 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 13:06:01,635 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 13:06:01,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 13:06:01,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 13:06:02,169 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 13:06:02,170 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 13:06:03.180 UTC [20938] WARNING: there is no transaction in progress
2026-08-08 13:06:03.196 UTC [20938] WARNING: there is no transaction in progress
2026-08-08 13:06:03.214 UTC [20938] WARNING: there is no transaction in progress
2026-08-08 13:06:03.230 UTC [20938] WARNING: there is no transaction in progress
2026-08-08 13:06:03.247 UTC [20938] WARNING: there is no transaction in progress
2026-08-08 13:06:03.266 UTC [20938] WARNING: there is no transaction in progress
2026-08-08 13:06:03.321 UTC [20938] WARNING: there is no transaction in progress
2026-08-08 13:06:03.328 UTC [20938] WARNING: there is no transaction in progress
2026-08-08 13:06:03.342 UTC [20938] WARNING: there is no transaction in progress
2026-08-08 13:06:03.352 UTC [20938] WARNING: there is no transaction in progress
2026-08-08 13:06:03.362 UTC [20938] WARNING: there is no transaction in progress
2026-08-08 13:06:03.376 UTC [20938] WARNING: there is no transaction in progress
2026-08-08 13:06:03.386 UTC [20938] WARNING: there is no transaction in progress
2026-08-08 13:06:03.392 UTC [20938] WARNING: there is no transaction in progress
2026-08-08 13:06:03.398 UTC [20938] WARNING: there is no transaction in progress
2026-08-08 13:06:03.405 UTC [20938] WARNING: there is no transaction in progress
2026-08-08 13:06:04,053 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 13:06:04,054 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 13:06:04,290 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 13:06:04,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 13:06:04,538 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 13:06:04,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 13:06:04,804 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 13:06:04,804 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 13:06:05,071 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 13:06:05,325 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-t3q3.c.gcp-upstream.internal';"
2026-08-08 13:06:05,564 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 13:06:05,564 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 13:06:05,821 [I] DB_PATCHES have already been applied
I20260808 13:06:05.827272 1522 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ranger-admin:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-task9J1ueI/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 13:06:05.827370 1522 mini_ranger.cc:193] Using host: 127.1.124.148
I20260808 13:06:05.849187 1522 mini_ranger.cc:227] Ranger admin URL: http://127.1.124.148:49187
Aug 08, 2026 1:06:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 1:06:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 1:06:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 1:06:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 1:06:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 1:06:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 1:06:06 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 13:06:07.299751 1522 mini_ranger.cc:160] Time spent starting Ranger: real 9.164s user 0.005s sys 0.057s
Aug 08, 2026 1:06:27 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 1:06:27 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 1:06:27 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 1:06:27 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 1:06:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 1:06:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 1:06:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 1:06:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 1:06:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 1:06:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 1:06:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 1:06:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 1:06:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 1:06:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 1:06:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 1:06:27 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 1:06:27 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 1:06:28 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 13:06:28.698405 1522 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.1.124.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.1.124.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.1.124.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/rangerkms_127.1.124.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.1.124.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/rangerkms_127.1.124.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.1.124.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.1.124.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.1.124.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/HTTP_127.1.124.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.1.124.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/HTTP_127.1.124.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 13:06:28 dist-test-slave-t3q3 krb5kdc[20619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194388, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 13:06:28.928154 1522 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 13:06:28.956113 1522 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 13:06:29.826747 1522 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 13:06:29.826902 1522 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ranger-kms
I20260808 13:06:29.826960 1522 mini_ranger_kms.cc:227] Using postgres at 127.1.124.148:54297
2026-08-08 13:06:30,252 [I] DB FLAVOR :POSTGRES
2026-08-08 13:06:30,253 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 13:06:30,253 [I] Checking connection
2026-08-08 13:06:30,253 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 13:06:30,529 [I] connection success
2026-08-08 13:06:30,529 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 13:06:30,529 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 13:06:30,529 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 13:06:30,827 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 13:06:30,827 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 13:06:30,828 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 13:06:30,828 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.1.124.148:54297/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 13:06:31,101 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 13:06:31.572456 1522 mini_ranger_kms.cc:310] Created kms service
I20260808 13:06:31.764994 1522 mini_ranger_kms.cc:326] Created kudu user
I20260808 13:06:31.811024 1522 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 13:06:31 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194388, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
I20260808 13:06:32.207172 1522 mini_ranger_kms.cc:384] Added ranger policy
I20260808 13:06:32.207504 1522 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ranger-kms:/tmp/dist-test-task9J1ueI/build/asan/bin/postgresql.jar:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-task9J1ueI/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 13:06:32.207571 1522 mini_ranger_kms.cc:240] Using host: 127.1.124.148
I20260808 13:06:32.230643 1522 mini_ranger_kms.cc:276] Ranger KMS PID: 21210
I20260808 13:06:32.230775 1522 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.1.124.148:43545
Aug 08, 2026 1:06:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 1:06:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 1:06:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 1:06:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 1:06:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 1:06:33 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 13:06:34 dist-test-slave-t3q3 krb5kdc[20619](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786194394, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.124.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:06:35.160866 1522 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.233s user 0.087s sys 0.374s
I20260808 13:06:35.161199 1522 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 13:06:35.161377 1522 mini_ranger_kms.cc:401] 127.1.124.148:43545/kms/v1/keys
Aug 08 13:06:35 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194394, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.124.148@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
Aug 08 13:06:35 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194394, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.1.124.148@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
Aug 08 13:06:38 dist-test-slave-t3q3 krb5kdc[20619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:06:38.124783 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:42435
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:33457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:42435
--ranger_config_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.124.148:43545/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:06:38.481540 21296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:06:38.481907 21296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:06:38.482074 21296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:06:38.489629 21296 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:06:38.489796 21296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:06:38.489899 21296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:06:38.490001 21296 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:06:38.490100 21296 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:06:38.500698 21296 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:33457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/wal
--ranger_kms_url=127.1.124.148:43545/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:42435
--ranger_config_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:42435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:06:38.502051 21296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:06:38.503499 21296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:06:38.516450 21306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:38.516544 21302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:38.516558 21303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:06:38.516746 21296 server_base.cc:1034] running on GCE node
I20260808 13:06:38.517831 21296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:06:38.519243 21296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:06:38.520485 21296 hybrid_clock.cc:648] HybridClock initialized: now 1786194398520441 us; error 36 us; skew 500 ppm
I20260808 13:06:38.520874 21296 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:06:38 dist-test-slave-t3q3 krb5kdc[20619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194398, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:06:38.524035 21296 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:06:38.525600 21296 webserver.cc:460] Webserver started at http://127.1.124.190:33733/ using document root <none> and password file <none>
I20260808 13:06:38.526263 21296 fs_manager.cc:359] Metadata directory not provided
I20260808 13:06:38.526410 21296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:06:38.526758 21296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 13:06:38 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194398, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
I20260808 13:06:38.592909 21296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "7a9fdf4b0ef141e1b0dc41b2d37ee916"
format_stamp: "Formatted at 2026-08-08 13:06:38 on dist-test-slave-t3q3"
server_key: "f897a87a51cb4a2a9ff3f48986fcb08e"
server_key_iv: "b47afb774166c6c41d1b5142506567b8"
server_key_version: "kuduclusterkey@0"
I20260808 13:06:38.593837 21296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "7a9fdf4b0ef141e1b0dc41b2d37ee916"
format_stamp: "Formatted at 2026-08-08 13:06:38 on dist-test-slave-t3q3"
server_key: "f897a87a51cb4a2a9ff3f48986fcb08e"
server_key_iv: "b47afb774166c6c41d1b5142506567b8"
server_key_version: "kuduclusterkey@0"
I20260808 13:06:38.599412 21296 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:06:38.618422 21316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:38.619905 21296 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260808 13:06:38.620179 21296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "7a9fdf4b0ef141e1b0dc41b2d37ee916"
format_stamp: "Formatted at 2026-08-08 13:06:38 on dist-test-slave-t3q3"
server_key: "f897a87a51cb4a2a9ff3f48986fcb08e"
server_key_iv: "b47afb774166c6c41d1b5142506567b8"
server_key_version: "kuduclusterkey@0"
I20260808 13:06:38.620438 21296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:06:38.661024 21296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:06:38.665270 21296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:06:38.665640 21296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:06:38.684784 21296 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:42435
I20260808 13:06:38.684808 21404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:42435 every 8 connection(s)
I20260808 13:06:38.686249 21296 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:06:38.690229 21405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:06:38.696616 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 21296
I20260808 13:06:38.696868 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:06:38.697778 1522 external_mini_cluster.cc:1428] Setting key d2bd82507be16000b5d9dea3acd69aa4
I20260808 13:06:38.702592 21405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916: Bootstrap starting.
I20260808 13:06:38.706590 21405 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916: Neither blocks nor log segments found. Creating new log.
I20260808 13:06:38.707727 21405 log.cc:824] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916: Log is configured to *not* fsync() on all Append() calls
I20260808 13:06:38.711717 21405 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916: No bootstrap required, opened a new log
Aug 08 13:06:38 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:06:38.723701 21405 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9fdf4b0ef141e1b0dc41b2d37ee916" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42435 } }
I20260808 13:06:38.724288 21405 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:06:38.724437 21405 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a9fdf4b0ef141e1b0dc41b2d37ee916, State: Initialized, Role: FOLLOWER
I20260808 13:06:38.725288 21405 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9fdf4b0ef141e1b0dc41b2d37ee916" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42435 } }
I20260808 13:06:38.725533 21410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:38.700493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46118 (local address 127.1.124.190:42435)
0808 13:06:38.701475 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:38.701489 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:38.701537 (+ 48us) server_negotiation.cc:407] Connection header received
0808 13:06:38.703549 (+ 2012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:38.703576 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:38.704142 (+ 566us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:38.704655 (+ 513us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:38.706493 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:38.708367 (+ 1874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:38.710050 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:38.711209 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:38.711415 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:38.715724 (+ 4309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:38.715737 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:38.715760 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:38.715821 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:38.720392 (+ 4571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:38.721121 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:38.721131 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:38.721143 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:38.721370 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:38.721754 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:38.721760 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:38.721764 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:38.722641 (+ 877us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:38.722961 (+ 320us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:38.723316 (+ 355us) server_negotiation.cc:299] Negotiation successful
0808 13:06:38.723817 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":746,"thread_start_us":479,"threads_started":1}
I20260808 13:06:38.725643 21405 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:06:38.725813 21405 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:06:38.726068 21405 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:06:38.729873 21405 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9fdf4b0ef141e1b0dc41b2d37ee916" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42435 } }
I20260808 13:06:38.730728 21405 leader_election.cc:302] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a9fdf4b0ef141e1b0dc41b2d37ee916; no voters:
I20260808 13:06:38.732360 21405 leader_election.cc:288] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:06:38.732506 21412 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:06:38.733565 21412 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [term 1 LEADER]: Becoming Leader. State: Replica: 7a9fdf4b0ef141e1b0dc41b2d37ee916, State: Running, Role: LEADER
I20260808 13:06:38.734242 21412 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9fdf4b0ef141e1b0dc41b2d37ee916" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42435 } }
I20260808 13:06:38.736446 21405 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:06:38.739292 21415 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a9fdf4b0ef141e1b0dc41b2d37ee916" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9fdf4b0ef141e1b0dc41b2d37ee916" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42435 } } }
I20260808 13:06:38.739683 21415 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [sys.catalog]: This master's current role is: LEADER
I20260808 13:06:38.739408 21413 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a9fdf4b0ef141e1b0dc41b2d37ee916. Latest consensus state: current_term: 1 leader_uuid: "7a9fdf4b0ef141e1b0dc41b2d37ee916" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9fdf4b0ef141e1b0dc41b2d37ee916" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42435 } } }
I20260808 13:06:38.739845 21413 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [sys.catalog]: This master's current role is: LEADER
I20260808 13:06:38.744834 21419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:06:38.751808 21419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:06:38.762526 21405 ranger_client.cc:318] Using new properties file: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 13:06:38.776510 21419 catalog_manager.cc:1347] Generated new cluster ID: 30e0994344e4462a8ce9eaca893f9c59
I20260808 13:06:38.776764 21419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:06:38.813932 21419 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:06:38.816286 21419 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:06:38.827461 21419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916: Generated new TSK 0
I20260808 13:06:38.828830 21419 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.124.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
I20260808 13:06:39.611449 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.124.148:43545/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42435
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:33457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 13:06:39 dist-test-slave-t3q3 krb5kdc[20619](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786194399, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 13:06:40.065093 21474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:06:40.065546 21474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:06:40.065771 21474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:06:40.077840 21474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:06:40.078019 21474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:06:40.078280 21474 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:06:40.094581 21474 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:33457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.1.124.148:43545/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:06:40.096539 21474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:06:40.098413 21474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:06:40.120007 21490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:40.120031 21489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:40.125001 21494 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:06:40.125851 21474 server_base.cc:1034] running on GCE node
I20260808 13:06:40.126739 21474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:06:40.132460 21474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:06:40.133755 21474 hybrid_clock.cc:648] HybridClock initialized: now 1786194400133695 us; error 45 us; skew 500 ppm
I20260808 13:06:40.134255 21474 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:06:40 dist-test-slave-t3q3 krb5kdc[20619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194400, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:06:40.138584 21474 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:06:40.140731 21474 webserver.cc:460] Webserver started at http://127.1.124.129:32901/ using document root <none> and password file <none>
I20260808 13:06:40.141731 21474 fs_manager.cc:359] Metadata directory not provided
I20260808 13:06:40.141886 21474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:06:40.142278 21474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 13:06:40 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194400, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
I20260808 13:06:40.161962 21474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "c37e05c9fa08490c8520dd4ffc2a83cf"
format_stamp: "Formatted at 2026-08-08 13:06:40 on dist-test-slave-t3q3"
server_key: "89eb5d9c5630575af5e9bf283bb39933"
server_key_iv: "62747b8901a70cba80ddccaccaa468b2"
server_key_version: "kuduclusterkey@0"
I20260808 13:06:40.163395 21474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "c37e05c9fa08490c8520dd4ffc2a83cf"
format_stamp: "Formatted at 2026-08-08 13:06:40 on dist-test-slave-t3q3"
server_key: "89eb5d9c5630575af5e9bf283bb39933"
server_key_iv: "62747b8901a70cba80ddccaccaa468b2"
server_key_version: "kuduclusterkey@0"
I20260808 13:06:40.177593 21474 fs_manager.cc:691] Time spent creating directory manager: real 0.014s user 0.008s sys 0.004s
I20260808 13:06:40.210949 21503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:40.213172 21474 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.005s sys 0.008s
I20260808 13:06:40.213385 21474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "c37e05c9fa08490c8520dd4ffc2a83cf"
format_stamp: "Formatted at 2026-08-08 13:06:40 on dist-test-slave-t3q3"
server_key: "89eb5d9c5630575af5e9bf283bb39933"
server_key_iv: "62747b8901a70cba80ddccaccaa468b2"
server_key_version: "kuduclusterkey@0"
I20260808 13:06:40.213652 21474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:06:40.321731 21474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:06:40.327270 21474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:06:40.327632 21474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:06:40.335155 21474 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:06:40.343465 21474 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:06:40.343580 21474 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:40.343729 21474 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:06:40.343788 21474 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 13:06:40 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194399, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
I20260808 13:06:40.887713 21682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:36743 every 8 connection(s)
I20260808 13:06:40.888286 21474 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:36743
I20260808 13:06:40.890381 21474 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:06:40.897953 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 21474
I20260808 13:06:40.898303 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:06:40.899369 1522 external_mini_cluster.cc:1428] Setting key a3c177b67c1a7d70dfc395021199b319
Aug 08 13:06:40 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194399, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
Aug 08 13:06:40 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194400, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.124.130@KRBTEST.COM; defaulting to no policy
I20260808 13:06:40.951124 21685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:40.904517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:38363 (local address 127.1.124.190:42435)
0808 13:06:40.910242 (+ 5725us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:40.910250 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:40.926737 (+ 16487us) server_negotiation.cc:407] Connection header received
0808 13:06:40.929926 (+ 3189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:40.929934 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:40.930052 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:40.930141 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:40.932714 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:40.934049 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:40.936791 (+ 2742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:40.937556 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:40.937672 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:40.942709 (+ 5037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:40.942717 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:40.942722 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:40.942767 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:40.946780 (+ 4013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:40.947873 (+ 1093us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:40.947879 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:40.947884 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:40.948096 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:40.948745 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:40.948751 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:40.948756 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:40.949125 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:40.949279 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:40.950410 (+ 1131us) server_negotiation.cc:299] Negotiation successful
0808 13:06:40.950684 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5588,"thread_start_us":6737,"threads_started":1}
I20260808 13:06:40.953083 21686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:40.904911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:42435 (local address 127.1.124.129:38363)
0808 13:06:40.926533 (+ 21622us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:40.926600 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:40.929561 (+ 2961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:40.930815 (+ 1254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:40.930830 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:40.931530 (+ 700us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:06:40.932485 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:40.932525 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:40.934293 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:40.934301 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:40.936589 (+ 2288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:40.936608 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:40.938038 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:40.938048 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:40.938386 (+ 338us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:40.939278 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:06:40.939313 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:06:40.942389 (+ 3076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:06:40.946969 (+ 4580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:40.946980 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:40.946998 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:40.947710 (+ 712us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:40.948273 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:40.948278 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:40.948282 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:40.948594 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:40.949386 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:06:40.949395 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:06:40.949862 (+ 467us) client_negotiation.cc:769] Sending connection context
0808 13:06:40.950914 (+ 1052us) client_negotiation.cc:240] Negotiation successful
0808 13:06:40.951527 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21281,"thread_start_us":21015,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.130@KRBTEST.COM" created.
I20260808 13:06:40.964880 21683 heartbeater.cc:347] Connected to a master server at 127.1.124.190:42435
I20260808 13:06:40.965294 21683 heartbeater.cc:464] Registering TS with master...
I20260808 13:06:40.966290 21683 heartbeater.cc:511] Master 127.1.124.190:42435 requested a full tablet report, sending...
I20260808 13:06:40.969422 21349 ts_manager.cc:194] Registered new tserver with Master: c37e05c9fa08490c8520dd4ffc2a83cf (127.1.124.129:36743)
I20260808 13:06:40.971832 21349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:38363
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
I20260808 13:06:41.000385 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.124.148:43545/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42435
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:33457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:06:41.459457 21694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:06:41.459784 21694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:06:41.459949 21694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:06:41.467283 21694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:06:41.467440 21694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:06:41.467623 21694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:06:41.477437 21694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:33457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.1.124.148:43545/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:06:41.478754 21694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:06:41.480078 21694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:06:41.493851 21712 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:41.493782 21713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:41.493736 21715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:06:41.494837 21694 server_base.cc:1034] running on GCE node
I20260808 13:06:41.495467 21694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:06:41.496526 21694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:06:41.497733 21694 hybrid_clock.cc:648] HybridClock initialized: now 1786194401497689 us; error 45 us; skew 500 ppm
I20260808 13:06:41.498067 21694 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:06:41 dist-test-slave-t3q3 krb5kdc[20619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194401, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:06:41.501650 21694 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:06:41.503283 21694 webserver.cc:460] Webserver started at http://127.1.124.130:33679/ using document root <none> and password file <none>
I20260808 13:06:41.504053 21694 fs_manager.cc:359] Metadata directory not provided
I20260808 13:06:41.504249 21694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:06:41.504597 21694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 13:06:41 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194401, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
I20260808 13:06:41.519764 21694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "0ed0b11d02ae46c1ab5bdd11adc6621b"
format_stamp: "Formatted at 2026-08-08 13:06:41 on dist-test-slave-t3q3"
server_key: "bea7f9edb7a75256aca74b7f1f3885c8"
server_key_iv: "a790e43c5f1f4b62226ed14d3f41fefe"
server_key_version: "kuduclusterkey@0"
I20260808 13:06:41.520803 21694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "0ed0b11d02ae46c1ab5bdd11adc6621b"
format_stamp: "Formatted at 2026-08-08 13:06:41 on dist-test-slave-t3q3"
server_key: "bea7f9edb7a75256aca74b7f1f3885c8"
server_key_iv: "a790e43c5f1f4b62226ed14d3f41fefe"
server_key_version: "kuduclusterkey@0"
I20260808 13:06:41.526734 21694 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:06:41.544546 21723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:41.546723 21694 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260808 13:06:41.546960 21694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "0ed0b11d02ae46c1ab5bdd11adc6621b"
format_stamp: "Formatted at 2026-08-08 13:06:41 on dist-test-slave-t3q3"
server_key: "bea7f9edb7a75256aca74b7f1f3885c8"
server_key_iv: "a790e43c5f1f4b62226ed14d3f41fefe"
server_key_version: "kuduclusterkey@0"
I20260808 13:06:41.547245 21694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:06:41.573083 21694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:06:41.577104 21694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:06:41.577373 21694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:06:41.578293 21694 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:06:41.579627 21694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:06:41.579704 21694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:41.579792 21694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:06:41.579839 21694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:41.615152 21694 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:37723
I20260808 13:06:41.615202 21892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:37723 every 8 connection(s)
I20260808 13:06:41.616621 21694 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:06:41.617285 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 21694
I20260808 13:06:41.617501 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:06:41.618129 1522 external_mini_cluster.cc:1428] Setting key 948dd3c79d8d787c868d61553512afe2
Aug 08 13:06:41 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194401, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:06:41.637872 21895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:41.619866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:57129 (local address 127.1.124.190:42435)
0808 13:06:41.620483 (+ 617us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:41.620491 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:41.621629 (+ 1138us) server_negotiation.cc:407] Connection header received
0808 13:06:41.623517 (+ 1888us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:41.623526 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:41.623618 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:41.623742 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:41.625360 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:41.626146 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:41.627698 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:41.628186 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:41.628333 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:41.631783 (+ 3450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:41.631788 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:41.631791 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:41.631823 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:41.634655 (+ 2832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:41.635391 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:41.635398 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:41.635401 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:41.635526 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:41.635957 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:41.635963 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:41.635966 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:41.636383 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:41.636537 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:41.637209 (+ 672us) server_negotiation.cc:299] Negotiation successful
0808 13:06:41.637475 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":336,"threads_started":1}
I20260808 13:06:41.638906 21896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:41.620578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:42435 (local address 127.1.124.130:57129)
0808 13:06:41.621429 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:41.621476 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:41.623186 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:41.623988 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:41.623997 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:41.624516 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:06:41.625191 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:41.625210 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:41.626318 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:41.626322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:41.627553 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:41.627565 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:41.628435 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:41.628442 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:41.628684 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:41.629395 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:06:41.629421 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:06:41.631528 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:06:41.634804 (+ 3276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:41.634814 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:41.634831 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:41.635260 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:41.635649 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:41.635654 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:41.635657 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:41.635843 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:41.636514 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:06:41.636522 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:06:41.636855 (+ 333us) client_negotiation.cc:769] Sending connection context
0808 13:06:41.637139 (+ 284us) client_negotiation.cc:240] Negotiation successful
0808 13:06:41.637748 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":583,"thread_start_us":360,"threads_started":1}
I20260808 13:06:41.640514 21893 heartbeater.cc:347] Connected to a master server at 127.1.124.190:42435
I20260808 13:06:41.640849 21893 heartbeater.cc:464] Registering TS with master...
I20260808 13:06:41.641585 21893 heartbeater.cc:511] Master 127.1.124.190:42435 requested a full tablet report, sending...
I20260808 13:06:41.643085 21349 ts_manager.cc:194] Registered new tserver with Master: 0ed0b11d02ae46c1ab5bdd11adc6621b (127.1.124.130:37723)
I20260808 13:06:41.643908 21349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:57129
WARNING: no policy specified for kudu/127.1.124.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
I20260808 13:06:41.683441 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.124.148:43545/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42435
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:33457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:06:41.975698 21683 heartbeater.cc:503] Master 127.1.124.190:42435 was elected leader, sending a full tablet report...
W20260808 13:06:42.033309 21899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:06:42.033591 21899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:06:42.033699 21899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:06:42.041235 21899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:06:42.041350 21899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:06:42.041518 21899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:06:42.051882 21899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:33457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.1.124.148:43545/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:06:42.053243 21899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:06:42.054530 21899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:06:42.067617 21912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:42.067719 21909 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:42.067754 21908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:06:42.068888 21899 server_base.cc:1034] running on GCE node
I20260808 13:06:42.069548 21899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:06:42.070546 21899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:06:42.071770 21899 hybrid_clock.cc:648] HybridClock initialized: now 1786194402071697 us; error 63 us; skew 500 ppm
I20260808 13:06:42.072165 21899 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:06:42 dist-test-slave-t3q3 krb5kdc[20619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194402, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:06:42.078661 21899 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:06:42.080206 21899 webserver.cc:460] Webserver started at http://127.1.124.131:40689/ using document root <none> and password file <none>
I20260808 13:06:42.080967 21899 fs_manager.cc:359] Metadata directory not provided
I20260808 13:06:42.081120 21899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:06:42.081437 21899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 13:06:42 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194402, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
I20260808 13:06:42.097500 21899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "60d9a44245624a7682ec4ae233c12b99"
format_stamp: "Formatted at 2026-08-08 13:06:42 on dist-test-slave-t3q3"
server_key: "becc1da9a65a2a6e8f5a63f90adaee06"
server_key_iv: "05327d796d296d5e7a2c66237b42f1a2"
server_key_version: "kuduclusterkey@0"
I20260808 13:06:42.098399 21899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "60d9a44245624a7682ec4ae233c12b99"
format_stamp: "Formatted at 2026-08-08 13:06:42 on dist-test-slave-t3q3"
server_key: "becc1da9a65a2a6e8f5a63f90adaee06"
server_key_iv: "05327d796d296d5e7a2c66237b42f1a2"
server_key_version: "kuduclusterkey@0"
I20260808 13:06:42.104045 21899 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 13:06:42.118462 21924 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:42.120191 21899 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:06:42.120453 21899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "60d9a44245624a7682ec4ae233c12b99"
format_stamp: "Formatted at 2026-08-08 13:06:42 on dist-test-slave-t3q3"
server_key: "becc1da9a65a2a6e8f5a63f90adaee06"
server_key_iv: "05327d796d296d5e7a2c66237b42f1a2"
server_key_version: "kuduclusterkey@0"
I20260808 13:06:42.120687 21899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:06:42.140384 21899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:06:42.144345 21899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:06:42.144737 21899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:06:42.145802 21899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:06:42.147194 21899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:06:42.147302 21899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:42.147457 21899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:06:42.147539 21899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:42.182888 21899 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:36381
I20260808 13:06:42.182927 22098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:36381 every 8 connection(s)
I20260808 13:06:42.184305 21899 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:06:42.192770 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 21899
I20260808 13:06:42.192996 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:06:42.193665 1522 external_mini_cluster.cc:1428] Setting key 94e637838c700044a57049d320f0c42c
Aug 08 13:06:42 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194402, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:06:42.205545 22101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:42.186886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:55911 (local address 127.1.124.190:42435)
0808 13:06:42.187339 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:42.187343 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:42.188372 (+ 1029us) server_negotiation.cc:407] Connection header received
0808 13:06:42.190113 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:42.190117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:42.190201 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:42.190303 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:42.191681 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:42.192603 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:42.194785 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:42.195482 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:42.195641 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:42.199496 (+ 3855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:42.199500 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:42.199504 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:42.199550 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:42.202314 (+ 2764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:42.203145 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:42.203151 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:42.203155 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:42.203295 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:42.203751 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:42.203755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:42.203757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:42.204012 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:42.204225 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:42.204935 (+ 710us) server_negotiation.cc:299] Negotiation successful
0808 13:06:42.205175 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":237,"threads_started":1}
I20260808 13:06:42.206503 22102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:42.187457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:42435 (local address 127.1.124.131:55911)
0808 13:06:42.188207 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:42.188256 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:42.189853 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:42.190508 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:42.190515 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:42.190937 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:06:42.191515 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:42.191532 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:42.192757 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:42.192763 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:42.194626 (+ 1863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:42.194638 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:42.195949 (+ 1311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:42.195959 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:42.196282 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:42.197082 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:06:42.197117 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:06:42.199270 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:06:42.202472 (+ 3202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:42.202481 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:42.202498 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:42.203021 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:42.203412 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:42.203417 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:42.203421 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:42.203604 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:42.204218 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:06:42.204226 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:06:42.204566 (+ 340us) client_negotiation.cc:769] Sending connection context
0808 13:06:42.204840 (+ 274us) client_negotiation.cc:240] Negotiation successful
0808 13:06:42.205379 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":278,"threads_started":1}
I20260808 13:06:42.208084 22099 heartbeater.cc:347] Connected to a master server at 127.1.124.190:42435
I20260808 13:06:42.208422 22099 heartbeater.cc:464] Registering TS with master...
I20260808 13:06:42.209331 22099 heartbeater.cc:511] Master 127.1.124.190:42435 requested a full tablet report, sending...
I20260808 13:06:42.210855 21349 ts_manager.cc:194] Registered new tserver with Master: 60d9a44245624a7682ec4ae233c12b99 (127.1.124.131:36381)
I20260808 13:06:42.211620 21349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:55911
I20260808 13:06:42.222298 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:06:42.224151 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 21474
Aug 08 13:06:42 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.129@KRBTEST.COM
I20260808 13:06:42.242014 22104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:42.225188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39286 (local address 127.1.124.129:36743)
0808 13:06:42.225911 (+ 723us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:42.225922 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:42.225954 (+ 32us) server_negotiation.cc:407] Connection header received
0808 13:06:42.226108 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:42.226115 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:42.226333 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:42.226519 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:42.228183 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:42.229247 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:42.230715 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:42.231296 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:42.231446 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:42.236029 (+ 4583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:42.236040 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:42.236099 (+ 59us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:42.236144 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:42.239036 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:42.239756 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:42.239763 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:42.239766 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:42.239903 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:42.240359 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:42.240366 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:42.240369 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:42.240724 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:42.241038 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:42.241375 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 13:06:42.241657 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540,"thread_start_us":327,"threads_started":1}
I20260808 13:06:42.243305 21659 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:06:42.326561 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 21474
I20260808 13:06:42.351533 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 21694
Aug 08 13:06:42 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.130@KRBTEST.COM
I20260808 13:06:42.373014 22110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:42.352577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47066 (local address 127.1.124.130:37723)
0808 13:06:42.353137 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:42.353147 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:42.353181 (+ 34us) server_negotiation.cc:407] Connection header received
0808 13:06:42.353257 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:42.353263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:42.353525 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:42.353661 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:42.355029 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:42.356038 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:42.357531 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:42.358091 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:42.358201 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:42.367171 (+ 8970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:42.367186 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:42.367202 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:42.367241 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:42.370394 (+ 3153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:42.371075 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:42.371083 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:42.371085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:42.371218 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:42.371564 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:42.371567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:42.371569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:42.371918 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:42.372230 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:42.372472 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 13:06:42.372697 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":285,"threads_started":1}
I20260808 13:06:42.374327 21884 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:06:42.457010 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 21694
I20260808 13:06:42.485808 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 21899
Aug 08 13:06:42 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.131@KRBTEST.COM
I20260808 13:06:42.501910 22112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:42.486765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52636 (local address 127.1.124.131:36381)
0808 13:06:42.487460 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:42.487466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:42.487495 (+ 29us) server_negotiation.cc:407] Connection header received
0808 13:06:42.487558 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:42.487564 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:42.487795 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:42.487924 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:42.489317 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:42.490315 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:42.491542 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:42.492201 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:42.492297 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:42.496773 (+ 4476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:42.496784 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:42.496798 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:42.496830 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:42.499394 (+ 2564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:42.500005 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:42.500012 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:42.500015 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:42.500205 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:42.500528 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:42.500532 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:42.500533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:42.500860 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:42.501103 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:42.501314 (+ 211us) server_negotiation.cc:299] Negotiation successful
0808 13:06:42.501509 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":369,"threads_started":1}
I20260808 13:06:42.503265 22091 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:06:42.586573 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 21899
I20260808 13:06:42.609843 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 21296
I20260808 13:06:42.611250 21399 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:06:42.718788 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 21296
I20260808 13:06:42.752823 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:42435
--webserver_interface=127.1.124.190
--webserver_port=33733
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:33457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:42435
--ranger_config_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.124.148:43545/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.124.148:43545/kms with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:06:43.100281 22117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:06:43.100667 22117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:06:43.100838 22117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:06:43.108613 22117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:06:43.108767 22117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:06:43.108875 22117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:06:43.108978 22117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:06:43.109069 22117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:06:43.119444 22117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:33457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.124.148:43545/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:42435
--ranger_config_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:42435
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=33733
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:06:43.120857 22117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:06:43.122162 22117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:06:43.134299 22128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:43.134382 22123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:43.134369 22124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:06:43.137773 22117 server_base.cc:1034] running on GCE node
I20260808 13:06:43.138609 22117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:06:43.140182 22117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:06:43.141417 22117 hybrid_clock.cc:648] HybridClock initialized: now 1786194403141362 us; error 56 us; skew 500 ppm
I20260808 13:06:43.141774 22117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:06:43 dist-test-slave-t3q3 krb5kdc[20619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194403, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:06:43.144991 22117 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:06:43.146461 22117 webserver.cc:460] Webserver started at http://127.1.124.190:33733/ using document root <none> and password file <none>
I20260808 13:06:43.147148 22117 fs_manager.cc:359] Metadata directory not provided
I20260808 13:06:43.147310 22117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 13:06:43 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194403, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
I20260808 13:06:43.197705 22117 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 13:06:43.200969 22142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:43.202354 22117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260808 13:06:43.202574 22117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "7a9fdf4b0ef141e1b0dc41b2d37ee916"
format_stamp: "Formatted at 2026-08-08 13:06:38 on dist-test-slave-t3q3"
server_key: "f897a87a51cb4a2a9ff3f48986fcb08e"
server_key_iv: "b47afb774166c6c41d1b5142506567b8"
server_key_version: "kuduclusterkey@0"
I20260808 13:06:43.203562 22117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:06:43.219048 22117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:06:43.222837 22117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:06:43.223145 22117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:06:43.241811 22117 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:42435
I20260808 13:06:43.241830 22228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:42435 every 8 connection(s)
I20260808 13:06:43.243418 22117 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:06:43.246469 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 22117
I20260808 13:06:43.247313 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.124.148:43545/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:36743
--local_ip_for_outbound_sockets=127.1.124.129
--tserver_master_addrs=127.1.124.190:42435
--webserver_port=32901
--webserver_interface=127.1.124.129
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:33457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.124.148:43545/kms with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:06:43.250334 22231 sys_catalog.cc:263] Verifying existing consensus state
I20260808 13:06:43.253580 22231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916: Bootstrap starting.
I20260808 13:06:43.277822 22231 log.cc:824] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916: Log is configured to *not* fsync() on all Append() calls
I20260808 13:06:43.287213 22231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 13:06:43.287792 22231 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916: Bootstrap complete.
I20260808 13:06:43.296944 22231 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9fdf4b0ef141e1b0dc41b2d37ee916" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42435 } }
I20260808 13:06:43.297899 22231 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a9fdf4b0ef141e1b0dc41b2d37ee916, State: Initialized, Role: FOLLOWER
I20260808 13:06:43.298491 22231 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9fdf4b0ef141e1b0dc41b2d37ee916" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42435 } }
I20260808 13:06:43.298748 22231 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:06:43.298882 22231 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:06:43.299065 22231 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:06:43.303085 22231 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9fdf4b0ef141e1b0dc41b2d37ee916" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42435 } }
I20260808 13:06:43.303576 22231 leader_election.cc:302] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a9fdf4b0ef141e1b0dc41b2d37ee916; no voters:
I20260808 13:06:43.304261 22231 leader_election.cc:288] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 13:06:43.304414 22237 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 13:06:43.306823 22231 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:06:43.311584 22237 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [term 2 LEADER]: Becoming Leader. State: Replica: 7a9fdf4b0ef141e1b0dc41b2d37ee916, State: Running, Role: LEADER
I20260808 13:06:43.312446 22237 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9fdf4b0ef141e1b0dc41b2d37ee916" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42435 } }
I20260808 13:06:43.315336 22238 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "7a9fdf4b0ef141e1b0dc41b2d37ee916" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9fdf4b0ef141e1b0dc41b2d37ee916" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42435 } } }
I20260808 13:06:43.315826 22238 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [sys.catalog]: This master's current role is: LEADER
I20260808 13:06:43.316461 22239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a9fdf4b0ef141e1b0dc41b2d37ee916. Latest consensus state: current_term: 2 leader_uuid: "7a9fdf4b0ef141e1b0dc41b2d37ee916" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a9fdf4b0ef141e1b0dc41b2d37ee916" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42435 } } }
I20260808 13:06:43.316779 22239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916 [sys.catalog]: This master's current role is: LEADER
I20260808 13:06:43.318908 22250 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:06:43.333580 22250 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:06:43.335553 22250 catalog_manager.cc:1259] Loaded cluster ID: 30e0994344e4462a8ce9eaca893f9c59
I20260808 13:06:43.335775 22250 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:06:43.341066 22250 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:06:43.343045 22250 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916: Loaded TSK: 0
I20260808 13:06:43.345122 22250 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:06:43.353724 22231 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260808 13:06:43.709633 22233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:06:43.710122 22233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:06:43.710327 22233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:06:43.722544 22233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:06:43.722718 22233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:06:43.722967 22233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:06:43.739411 22233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:33457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.124.148:43545/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:36743
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=32901
--tserver_master_addrs=127.1.124.190:42435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:06:43.741259 22233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:06:43.743150 22233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:06:43.789120 22289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:43.790001 22291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:06:43.797968 22233 server_base.cc:1034] running on GCE node
W20260808 13:06:43.798895 22297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:06:43.799886 22233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:06:43.807494 22233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:06:43.808959 22233 hybrid_clock.cc:648] HybridClock initialized: now 1786194403808876 us; error 87 us; skew 500 ppm
I20260808 13:06:43.809445 22233 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:06:43 dist-test-slave-t3q3 krb5kdc[20619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194403, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:06:43.814136 22233 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:06:43.817389 22233 webserver.cc:460] Webserver started at http://127.1.124.129:32901/ using document root <none> and password file <none>
I20260808 13:06:43.819730 22233 fs_manager.cc:359] Metadata directory not provided
I20260808 13:06:43.820032 22233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 13:06:43 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194403, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
I20260808 13:06:43.882421 22233 fs_manager.cc:709] Time spent opening directory manager: real 0.020s user 0.009s sys 0.008s
I20260808 13:06:43.886427 22307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:43.888185 22233 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260808 13:06:43.888391 22233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "c37e05c9fa08490c8520dd4ffc2a83cf"
format_stamp: "Formatted at 2026-08-08 13:06:40 on dist-test-slave-t3q3"
server_key: "89eb5d9c5630575af5e9bf283bb39933"
server_key_iv: "62747b8901a70cba80ddccaccaa468b2"
server_key_version: "kuduclusterkey@0"
I20260808 13:06:43.889665 22233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:06:43.954821 22233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:06:43.960469 22233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:06:43.960858 22233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:06:43.966312 22233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:06:43.970573 22233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:06:43.970805 22233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260808 13:06:43.971032 22233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:06:43.971199 22233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:44.060564 22233 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:36743
I20260808 13:06:44.060585 22471 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:36743 every 8 connection(s)
I20260808 13:06:44.062662 22233 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:06:44.067094 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 22233
I20260808 13:06:44.068277 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.124.148:43545/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:37723
--local_ip_for_outbound_sockets=127.1.124.130
--tserver_master_addrs=127.1.124.190:42435
--webserver_port=33679
--webserver_interface=127.1.124.130
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:33457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.124.148:43545/kms with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 13:06:44 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194403, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:06:44.105610 22480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:44.066508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:42435 (local address 127.1.124.129:52937)
0808 13:06:44.078676 (+ 12168us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:44.078752 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:44.082127 (+ 3375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:44.084285 (+ 2158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:44.084296 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:44.084978 (+ 682us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:06:44.085598 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:44.085617 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:44.087727 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:44.087734 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:44.089843 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:44.089858 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:44.091689 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:44.091697 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:44.091971 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:44.092780 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:06:44.092804 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:06:44.095706 (+ 2902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:06:44.099973 (+ 4267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:44.099983 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:44.100001 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:44.100679 (+ 678us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:44.101258 (+ 579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:44.101263 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:44.101267 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:44.101500 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:44.102650 (+ 1150us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:06:44.102658 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:06:44.103112 (+ 454us) client_negotiation.cc:769] Sending connection context
0808 13:06:44.103496 (+ 384us) client_negotiation.cc:240] Negotiation successful
0808 13:06:44.104141 (+ 645us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11781,"thread_start_us":11519,"threads_started":1}
I20260808 13:06:44.105611 22478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:44.066518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:52937 (local address 127.1.124.190:42435)
0808 13:06:44.080673 (+ 14155us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:44.080692 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:44.080757 (+ 65us) server_negotiation.cc:407] Connection header received
0808 13:06:44.083005 (+ 2248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:44.083028 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:44.083604 (+ 576us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:44.084095 (+ 491us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:44.085843 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:44.087530 (+ 1687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:44.090004 (+ 2474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:44.091068 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:44.091267 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:44.096105 (+ 4838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:44.096120 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:44.096143 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:44.096233 (+ 90us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:44.099759 (+ 3526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:44.100861 (+ 1102us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:44.100870 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:44.100878 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:44.101092 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:44.101645 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:44.101650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:44.101654 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:44.102347 (+ 693us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:44.102564 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:44.103748 (+ 1184us) server_negotiation.cc:299] Negotiation successful
0808 13:06:44.104190 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13857,"thread_start_us":13636,"threads_started":1}
I20260808 13:06:44.109817 22474 heartbeater.cc:347] Connected to a master server at 127.1.124.190:42435
I20260808 13:06:44.110155 22474 heartbeater.cc:464] Registering TS with master...
I20260808 13:06:44.110988 22474 heartbeater.cc:511] Master 127.1.124.190:42435 requested a full tablet report, sending...
W20260808 13:06:44.114629 22474 heartbeater.cc:644] Failed to heartbeat to 127.1.124.190:42435 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 13:06:44.114874 22474 heartbeater.cc:464] Registering TS with master...
I20260808 13:06:44.115183 22474 heartbeater.cc:511] Master 127.1.124.190:42435 requested a full tablet report, sending...
I20260808 13:06:44.116559 22474 heartbeater.cc:464] Registering TS with master...
I20260808 13:06:44.116900 22474 heartbeater.cc:511] Master 127.1.124.190:42435 requested a full tablet report, sending...
W20260808 13:06:44.118023 22474 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 13:06:44.517231 22481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:06:44.517531 22481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:06:44.517722 22481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:06:44.529440 22481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:06:44.529701 22481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:06:44.529984 22481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:06:44.544920 22481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:33457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.124.148:43545/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:37723
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=33679
--tserver_master_addrs=127.1.124.190:42435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:06:44.546339 22481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:06:44.547747 22481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:06:44.565764 22504 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:44.566435 22503 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:06:44.575603 22481 server_base.cc:1034] running on GCE node
W20260808 13:06:44.576141 22506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:06:44.577157 22481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:06:44.578442 22481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:06:44.579710 22481 hybrid_clock.cc:648] HybridClock initialized: now 1786194404579644 us; error 72 us; skew 500 ppm
I20260808 13:06:44.580181 22481 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:06:44 dist-test-slave-t3q3 krb5kdc[20619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194404, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:06:44.584045 22481 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:06:44.586212 22481 webserver.cc:460] Webserver started at http://127.1.124.130:33679/ using document root <none> and password file <none>
I20260808 13:06:44.586983 22481 fs_manager.cc:359] Metadata directory not provided
I20260808 13:06:44.587160 22481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 13:06:44 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194404, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
I20260808 13:06:44.609831 22481 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 13:06:44.612931 22523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:44.614435 22481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:06:44.614642 22481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "0ed0b11d02ae46c1ab5bdd11adc6621b"
format_stamp: "Formatted at 2026-08-08 13:06:41 on dist-test-slave-t3q3"
server_key: "bea7f9edb7a75256aca74b7f1f3885c8"
server_key_iv: "a790e43c5f1f4b62226ed14d3f41fefe"
server_key_version: "kuduclusterkey@0"
I20260808 13:06:44.615717 22481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 08 13:06:44 dist-test-slave-t3q3 krb5kdc[20619](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786194404, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:06:44.650182 22481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:06:44.655876 22481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:06:44.656260 22481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:06:44.657509 22481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:06:44.659118 22481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:06:44.659242 22481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:44.659394 22481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:06:44.659484 22481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:44.704468 22481 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:37723
I20260808 13:06:44.704514 22682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:37723 every 8 connection(s)
I20260808 13:06:44.706421 22481 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:06:44.714784 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 22481
I20260808 13:06:44.715783 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.1.124.148:43545/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:36381
--local_ip_for_outbound_sockets=127.1.124.131
--tserver_master_addrs=127.1.124.190:42435
--webserver_port=40689
--webserver_interface=127.1.124.131
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:33457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.1.124.148:43545/kms with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 13:06:44 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194404, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:06:44.757652 22687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:44.711130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:51939 (local address 127.1.124.190:42435)
0808 13:06:44.732882 (+ 21752us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:44.732889 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:44.733967 (+ 1078us) server_negotiation.cc:407] Connection header received
0808 13:06:44.740713 (+ 6746us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:44.740718 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:44.740798 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:44.740949 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:44.742553 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:44.743330 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:44.745769 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:44.746432 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:44.746567 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:44.750798 (+ 4231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:44.750812 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:44.750816 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:44.750850 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:44.753980 (+ 3130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:44.754835 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:44.754841 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:44.754844 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:44.754978 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:44.755647 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:44.755652 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:44.755655 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:44.756014 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:44.756217 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:44.757010 (+ 793us) server_negotiation.cc:299] Negotiation successful
0808 13:06:44.757258 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21570,"thread_start_us":21782,"threads_started":1}
I20260808 13:06:44.758785 22685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:44.710513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:42435 (local address 127.1.124.130:51939)
0808 13:06:44.733710 (+ 23197us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:44.733800 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:44.736598 (+ 2798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:44.741256 (+ 4658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:44.741266 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:44.741735 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:06:44.742355 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:44.742374 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:44.743506 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:44.743512 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:44.745558 (+ 2046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:44.745576 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:44.746894 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:44.746903 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:44.747219 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:44.747985 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:06:44.748009 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:06:44.750543 (+ 2534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:06:44.754195 (+ 3652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:44.754204 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:44.754224 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:44.754692 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:44.755175 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:44.755182 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:44.755188 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:44.755480 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:44.756286 (+ 806us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:06:44.756292 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:06:44.756611 (+ 319us) client_negotiation.cc:769] Sending connection context
0808 13:06:44.756898 (+ 287us) client_negotiation.cc:240] Negotiation successful
0808 13:06:44.757465 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22802,"thread_start_us":26455,"threads_started":1}
I20260808 13:06:44.760406 22683 heartbeater.cc:347] Connected to a master server at 127.1.124.190:42435
I20260808 13:06:44.760766 22683 heartbeater.cc:464] Registering TS with master...
I20260808 13:06:44.761703 22683 heartbeater.cc:511] Master 127.1.124.190:42435 requested a full tablet report, sending...
I20260808 13:06:44.763899 22169 ts_manager.cc:194] Registered new tserver with Master: 0ed0b11d02ae46c1ab5bdd11adc6621b (127.1.124.130:37723)
I20260808 13:06:44.766179 22169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:51939
Aug 08 13:06:45 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194404, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
I20260808 13:06:45.118212 22474 heartbeater.cc:464] Registering TS with master...
I20260808 13:06:45.118662 22474 heartbeater.cc:511] Master 127.1.124.190:42435 requested a full tablet report, sending...
I20260808 13:06:45.119977 22169 ts_manager.cc:194] Registered new tserver with Master: c37e05c9fa08490c8520dd4ffc2a83cf (127.1.124.129:36743)
I20260808 13:06:45.121272 22169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:52937
W20260808 13:06:45.239602 22688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:06:45.240176 22688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:06:45.240391 22688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:06:45.253082 22688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:06:45.253319 22688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:06:45.253556 22688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:06:45.269596 22688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:33457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.1.124.148:43545/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:36381
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=40689
--tserver_master_addrs=127.1.124.190:42435
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:06:45.271792 22688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:06:45.273787 22688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:06:45.319903 22694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:45.320420 22693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:06:45.324592 22688 server_base.cc:1034] running on GCE node
W20260808 13:06:45.325058 22699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:06:45.326002 22688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:06:45.332652 22688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:06:45.334122 22688 hybrid_clock.cc:648] HybridClock initialized: now 1786194405334043 us; error 84 us; skew 500 ppm
I20260808 13:06:45.334625 22688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:06:45 dist-test-slave-t3q3 krb5kdc[20619](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194405, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:06:45.339425 22688 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:06:45.364204 22688 webserver.cc:460] Webserver started at http://127.1.124.131:40689/ using document root <none> and password file <none>
I20260808 13:06:45.365417 22688 fs_manager.cc:359] Metadata directory not provided
I20260808 13:06:45.365688 22688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 13:06:45 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194405, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
I20260808 13:06:45.428853 22688 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.006s sys 0.004s
I20260808 13:06:45.442329 22708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:45.447608 22688 fs_manager.cc:725] Time spent opening block manager: real 0.017s user 0.002s sys 0.011s
I20260808 13:06:45.447918 22688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "60d9a44245624a7682ec4ae233c12b99"
format_stamp: "Formatted at 2026-08-08 13:06:42 on dist-test-slave-t3q3"
server_key: "becc1da9a65a2a6e8f5a63f90adaee06"
server_key_iv: "05327d796d296d5e7a2c66237b42f1a2"
server_key_version: "kuduclusterkey@0"
I20260808 13:06:45.449517 22688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 08 13:06:45 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194404, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for HTTP/127.1.124.148@KRBTEST.COM
I20260808 13:06:45.502264 22688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:06:45.508761 22688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:06:45.509151 22688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:06:45.529385 22688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:06:45.531935 22688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:06:45.532150 22688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260808 13:06:45.532315 22688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:06:45.532428 22688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:45.771296 22683 heartbeater.cc:503] Master 127.1.124.190:42435 was elected leader, sending a full tablet report...
I20260808 13:06:45.863267 22688 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:36381
I20260808 13:06:45.863318 22884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:36381 every 8 connection(s)
I20260808 13:06:45.864928 22688 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:06:45.869517 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 22688
Aug 08 13:06:45 dist-test-slave-t3q3 krb5kdc[20619](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194405, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:06:45.894935 22894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:45.867736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:59389 (local address 127.1.124.190:42435)
0808 13:06:45.868274 (+ 538us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:45.868282 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:45.869955 (+ 1673us) server_negotiation.cc:407] Connection header received
0808 13:06:45.873467 (+ 3512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:45.873477 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:45.873617 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:45.873744 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:45.876455 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:45.878115 (+ 1660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:45.881334 (+ 3219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:45.882332 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:45.882459 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:45.887873 (+ 5414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:45.887879 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:45.887882 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:45.887927 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:45.891301 (+ 3374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:45.892130 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:45.892142 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:45.892145 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:45.892275 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:45.892773 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:45.892777 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:45.892780 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:45.893142 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:45.893300 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:45.894278 (+ 978us) server_negotiation.cc:299] Negotiation successful
0808 13:06:45.894552 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":232,"threads_started":1}
I20260808 13:06:45.896000 22896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:45.868352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:42435 (local address 127.1.124.131:59389)
0808 13:06:45.869800 (+ 1448us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:45.869839 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:45.872906 (+ 3067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:45.874339 (+ 1433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:45.874356 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:45.875160 (+ 804us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:06:45.876190 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:45.876219 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:45.878384 (+ 2165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:45.878394 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:45.881100 (+ 2706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:45.881122 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:45.882961 (+ 1839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:45.882971 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:45.883391 (+ 420us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:45.884493 (+ 1102us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:06:45.884548 (+ 55us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:06:45.887648 (+ 3100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:06:45.891446 (+ 3798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:45.891456 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:45.891474 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:45.891974 (+ 500us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:45.892392 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:45.892397 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:45.892401 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:45.892612 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:45.893378 (+ 766us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:06:45.893386 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:06:45.893832 (+ 446us) client_negotiation.cc:769] Sending connection context
0808 13:06:45.894162 (+ 330us) client_negotiation.cc:240] Negotiation successful
0808 13:06:45.894742 (+ 580us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":303,"threads_started":1}
I20260808 13:06:45.897429 22885 heartbeater.cc:347] Connected to a master server at 127.1.124.190:42435
I20260808 13:06:45.897781 22885 heartbeater.cc:464] Registering TS with master...
I20260808 13:06:45.898680 22885 heartbeater.cc:511] Master 127.1.124.190:42435 requested a full tablet report, sending...
I20260808 13:06:45.900377 22169 ts_manager.cc:194] Registered new tserver with Master: 60d9a44245624a7682ec4ae233c12b99 (127.1.124.131:36381)
I20260808 13:06:45.901556 22169 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:59389
I20260808 13:06:45.905867 22897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:45.870556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47072 (local address 127.1.124.190:42435)
0808 13:06:45.885765 (+ 15209us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:45.885772 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:45.886424 (+ 652us) server_negotiation.cc:407] Connection header received
0808 13:06:45.886489 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:45.886495 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:45.886588 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:45.886714 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:45.888979 (+ 2265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:45.890357 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:45.892770 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:45.893441 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:45.893554 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:45.899264 (+ 5710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:45.899269 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:45.899273 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:45.899302 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:45.903018 (+ 3716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:45.903923 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:45.903929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:45.903932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:45.904105 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:45.904544 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:45.904549 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:45.904552 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:45.904844 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:45.905049 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:45.905422 (+ 373us) server_negotiation.cc:299] Negotiation successful
0808 13:06:45.905585 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15087,"thread_start_us":15741,"threads_started":1}
I20260808 13:06:45.908788 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:06:45.927132 22897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:45.912194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47076 (local address 127.1.124.190:42435)
0808 13:06:45.912383 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:45.912389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:45.912877 (+ 488us) server_negotiation.cc:407] Connection header received
0808 13:06:45.913017 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:45.913022 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:45.913108 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:45.913227 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:45.915204 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:45.916266 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:45.918126 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:45.918578 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:45.918683 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:45.921904 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:45.921909 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:45.921912 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:45.921943 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:45.924622 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:45.925367 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:45.925370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:45.925372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:45.925497 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:45.925882 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:45.925885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:45.925887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:45.926128 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:45.926299 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:45.926661 (+ 362us) server_negotiation.cc:299] Negotiation successful
0808 13:06:45.926846 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 13:06:45.932580 22169 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:06:45.940186 22169 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:06:45.973165 22919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:45.956611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47074 (local address 127.1.124.130:37723)
0808 13:06:45.957367 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:45.957381 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:45.960653 (+ 3272us) server_negotiation.cc:407] Connection header received
0808 13:06:45.960852 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:45.960859 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:45.961116 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:45.961323 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:06:45.963334 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:45.965126 (+ 1792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:45.968820 (+ 3694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:45.971427 (+ 2607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:45.971536 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:45.971791 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:45.972545 (+ 754us) server_negotiation.cc:299] Negotiation successful
0808 13:06:45.972766 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":293,"threads_started":1}
I20260808 13:06:45.973858 22916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:45.954751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:36743 (local address 127.0.0.1:39298)
0808 13:06:45.955592 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:45.955644 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:45.955865 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:45.961610 (+ 5745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:45.961618 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:45.961665 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:06:45.962215 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:45.962235 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:45.965365 (+ 3130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:45.965369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:45.969302 (+ 3933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:45.969319 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:45.972885 (+ 3566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:45.972893 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:45.973089 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:45.973121 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 13:06:45.973221 (+ 100us) client_negotiation.cc:240] Negotiation successful
0808 13:06:45.973422 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":553,"mutex_wait_us":97,"spinlock_wait_cycles":896,"thread_start_us":403,"threads_started":1}
I20260808 13:06:45.974740 22917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:45.955040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39298 (local address 127.1.124.129:36743)
0808 13:06:45.960677 (+ 5637us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:45.960687 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:45.960711 (+ 24us) server_negotiation.cc:407] Connection header received
0808 13:06:45.960843 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:45.960851 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:45.961118 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:45.961323 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:06:45.962476 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:45.965208 (+ 2732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:45.969586 (+ 4378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:45.971427 (+ 1841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:45.971536 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:45.974172 (+ 2636us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:45.974257 (+ 85us) server_negotiation.cc:299] Negotiation successful
0808 13:06:45.974432 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5454,"thread_start_us":5317,"threads_started":1}
I20260808 13:06:45.974764 22918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:45.956096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:37723 (local address 127.0.0.1:47074)
0808 13:06:45.960524 (+ 4428us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:45.960550 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:45.960688 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:45.961610 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:45.961618 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:45.961671 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:06:45.962620 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:45.962633 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:45.965364 (+ 2731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:45.965369 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:45.968447 (+ 3078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:45.968463 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:45.972005 (+ 3542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:45.972012 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:45.972252 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:45.972307 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 13:06:45.973705 (+ 1398us) client_negotiation.cc:240] Negotiation successful
0808 13:06:45.973858 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4286,"mutex_wait_us":711,"spinlock_wait_cycles":15488,"thread_start_us":4155,"threads_started":1}
I20260808 13:06:45.979137 22383 tablet_service.cc:1467] Processing CreateTablet for tablet a3787f2dd4574dd29a66c75f38b3c81b (DEFAULT_TABLE table=test-table [id=97fef078284742fdb778924793e7bed8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:06:45.979382 22585 tablet_service.cc:1467] Processing CreateTablet for tablet a3787f2dd4574dd29a66c75f38b3c81b (DEFAULT_TABLE table=test-table [id=97fef078284742fdb778924793e7bed8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:06:45.981083 22585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3787f2dd4574dd29a66c75f38b3c81b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:06:45.982187 22383 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3787f2dd4574dd29a66c75f38b3c81b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:06:45.985213 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:45.961865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:36381 (local address 127.0.0.1:52644)
0808 13:06:45.968708 (+ 6843us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:45.968734 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:45.968892 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:45.977905 (+ 9013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:45.977911 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:45.977938 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:06:45.978258 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:45.978271 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:45.980397 (+ 2126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:45.980401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:45.982452 (+ 2051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:45.982469 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:45.984413 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:45.984420 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:45.984596 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:45.984633 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 13:06:45.984716 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 13:06:45.984862 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6652,"thread_start_us":6695,"threads_started":1}
I20260808 13:06:45.986546 22920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:45.960998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52644 (local address 127.1.124.131:36381)
0808 13:06:45.977284 (+ 16286us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:45.977293 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:45.977322 (+ 29us) server_negotiation.cc:407] Connection header received
0808 13:06:45.977417 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:45.977425 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:45.977671 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:45.978459 (+ 788us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:06:45.978764 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:45.980245 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:45.982689 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:45.984165 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:45.985737 (+ 1572us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:45.985974 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:45.986042 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 13:06:45.986212 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16104,"threads_started":1}
I20260808 13:06:45.990653 22785 tablet_service.cc:1467] Processing CreateTablet for tablet a3787f2dd4574dd29a66c75f38b3c81b (DEFAULT_TABLE table=test-table [id=97fef078284742fdb778924793e7bed8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:06:45.992169 22785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3787f2dd4574dd29a66c75f38b3c81b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:06:46.003219 22923 tablet_bootstrap.cc:492] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b: Bootstrap starting.
I20260808 13:06:46.009472 22923 tablet_bootstrap.cc:654] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b: Neither blocks nor log segments found. Creating new log.
I20260808 13:06:46.010918 22924 tablet_bootstrap.cc:492] T a3787f2dd4574dd29a66c75f38b3c81b P c37e05c9fa08490c8520dd4ffc2a83cf: Bootstrap starting.
I20260808 13:06:46.011036 22923 log.cc:824] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b: Log is configured to *not* fsync() on all Append() calls
I20260808 13:06:46.012825 22927 tablet_bootstrap.cc:492] T a3787f2dd4574dd29a66c75f38b3c81b P 60d9a44245624a7682ec4ae233c12b99: Bootstrap starting.
I20260808 13:06:46.015131 22924 tablet_bootstrap.cc:654] T a3787f2dd4574dd29a66c75f38b3c81b P c37e05c9fa08490c8520dd4ffc2a83cf: Neither blocks nor log segments found. Creating new log.
I20260808 13:06:46.016008 22927 tablet_bootstrap.cc:654] T a3787f2dd4574dd29a66c75f38b3c81b P 60d9a44245624a7682ec4ae233c12b99: Neither blocks nor log segments found. Creating new log.
I20260808 13:06:46.016438 22923 tablet_bootstrap.cc:492] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b: No bootstrap required, opened a new log
I20260808 13:06:46.016506 22924 log.cc:824] T a3787f2dd4574dd29a66c75f38b3c81b P c37e05c9fa08490c8520dd4ffc2a83cf: Log is configured to *not* fsync() on all Append() calls
I20260808 13:06:46.016750 22923 ts_tablet_manager.cc:1397] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.003s
I20260808 13:06:46.017272 22927 log.cc:824] T a3787f2dd4574dd29a66c75f38b3c81b P 60d9a44245624a7682ec4ae233c12b99: Log is configured to *not* fsync() on all Append() calls
I20260808 13:06:46.019745 22924 tablet_bootstrap.cc:492] T a3787f2dd4574dd29a66c75f38b3c81b P c37e05c9fa08490c8520dd4ffc2a83cf: No bootstrap required, opened a new log
I20260808 13:06:46.020040 22924 ts_tablet_manager.cc:1397] T a3787f2dd4574dd29a66c75f38b3c81b P c37e05c9fa08490c8520dd4ffc2a83cf: Time spent bootstrapping tablet: real 0.009s user 0.009s sys 0.000s
I20260808 13:06:46.022109 22927 tablet_bootstrap.cc:492] T a3787f2dd4574dd29a66c75f38b3c81b P 60d9a44245624a7682ec4ae233c12b99: No bootstrap required, opened a new log
I20260808 13:06:46.022305 22927 ts_tablet_manager.cc:1397] T a3787f2dd4574dd29a66c75f38b3c81b P 60d9a44245624a7682ec4ae233c12b99: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260808 13:06:46.028034 22923 raft_consensus.cc:348] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37e05c9fa08490c8520dd4ffc2a83cf" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 36743 } } peers { permanent_uuid: "0ed0b11d02ae46c1ab5bdd11adc6621b" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37723 } } peers { permanent_uuid: "60d9a44245624a7682ec4ae233c12b99" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 36381 } }
I20260808 13:06:46.028610 22923 raft_consensus.cc:374] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:06:46.028723 22923 raft_consensus.cc:729] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ed0b11d02ae46c1ab5bdd11adc6621b, State: Initialized, Role: FOLLOWER
I20260808 13:06:46.029539 22923 consensus_queue.cc:260] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37e05c9fa08490c8520dd4ffc2a83cf" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 36743 } } peers { permanent_uuid: "0ed0b11d02ae46c1ab5bdd11adc6621b" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37723 } } peers { permanent_uuid: "60d9a44245624a7682ec4ae233c12b99" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 36381 } }
I20260808 13:06:46.029794 22927 raft_consensus.cc:348] T a3787f2dd4574dd29a66c75f38b3c81b P 60d9a44245624a7682ec4ae233c12b99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37e05c9fa08490c8520dd4ffc2a83cf" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 36743 } } peers { permanent_uuid: "0ed0b11d02ae46c1ab5bdd11adc6621b" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37723 } } peers { permanent_uuid: "60d9a44245624a7682ec4ae233c12b99" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 36381 } }
I20260808 13:06:46.030314 22927 raft_consensus.cc:374] T a3787f2dd4574dd29a66c75f38b3c81b P 60d9a44245624a7682ec4ae233c12b99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:06:46.030471 22927 raft_consensus.cc:729] T a3787f2dd4574dd29a66c75f38b3c81b P 60d9a44245624a7682ec4ae233c12b99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60d9a44245624a7682ec4ae233c12b99, State: Initialized, Role: FOLLOWER
I20260808 13:06:46.031059 22924 raft_consensus.cc:348] T a3787f2dd4574dd29a66c75f38b3c81b P c37e05c9fa08490c8520dd4ffc2a83cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37e05c9fa08490c8520dd4ffc2a83cf" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 36743 } } peers { permanent_uuid: "0ed0b11d02ae46c1ab5bdd11adc6621b" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37723 } } peers { permanent_uuid: "60d9a44245624a7682ec4ae233c12b99" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 36381 } }
I20260808 13:06:46.031301 22927 consensus_queue.cc:260] T a3787f2dd4574dd29a66c75f38b3c81b P 60d9a44245624a7682ec4ae233c12b99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37e05c9fa08490c8520dd4ffc2a83cf" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 36743 } } peers { permanent_uuid: "0ed0b11d02ae46c1ab5bdd11adc6621b" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37723 } } peers { permanent_uuid: "60d9a44245624a7682ec4ae233c12b99" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 36381 } }
I20260808 13:06:46.031635 22924 raft_consensus.cc:374] T a3787f2dd4574dd29a66c75f38b3c81b P c37e05c9fa08490c8520dd4ffc2a83cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:06:46.031744 22923 ts_tablet_manager.cc:1428] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b: Time spent starting tablet: real 0.015s user 0.008s sys 0.006s
I20260808 13:06:46.031800 22924 raft_consensus.cc:729] T a3787f2dd4574dd29a66c75f38b3c81b P c37e05c9fa08490c8520dd4ffc2a83cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c37e05c9fa08490c8520dd4ffc2a83cf, State: Initialized, Role: FOLLOWER
I20260808 13:06:46.032622 22924 consensus_queue.cc:260] T a3787f2dd4574dd29a66c75f38b3c81b P c37e05c9fa08490c8520dd4ffc2a83cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37e05c9fa08490c8520dd4ffc2a83cf" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 36743 } } peers { permanent_uuid: "0ed0b11d02ae46c1ab5bdd11adc6621b" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37723 } } peers { permanent_uuid: "60d9a44245624a7682ec4ae233c12b99" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 36381 } }
I20260808 13:06:46.032899 22885 heartbeater.cc:503] Master 127.1.124.190:42435 was elected leader, sending a full tablet report...
I20260808 13:06:46.033550 22927 ts_tablet_manager.cc:1428] T a3787f2dd4574dd29a66c75f38b3c81b P 60d9a44245624a7682ec4ae233c12b99: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260808 13:06:46.034515 22474 heartbeater.cc:503] Master 127.1.124.190:42435 was elected leader, sending a full tablet report...
I20260808 13:06:46.035211 22924 ts_tablet_manager.cc:1428] T a3787f2dd4574dd29a66c75f38b3c81b P c37e05c9fa08490c8520dd4ffc2a83cf: Time spent starting tablet: real 0.015s user 0.002s sys 0.011s
W20260808 13:06:46.067242 22475 tablet.cc:2369] T a3787f2dd4574dd29a66c75f38b3c81b P c37e05c9fa08490c8520dd4ffc2a83cf: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:06:46.093865 22931 raft_consensus.cc:482] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:06:46.094327 22931 raft_consensus.cc:504] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37e05c9fa08490c8520dd4ffc2a83cf" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 36743 } } peers { permanent_uuid: "0ed0b11d02ae46c1ab5bdd11adc6621b" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37723 } } peers { permanent_uuid: "60d9a44245624a7682ec4ae233c12b99" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 36381 } }
I20260808 13:06:46.096426 22931 leader_election.cc:288] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c37e05c9fa08490c8520dd4ffc2a83cf (127.1.124.129:36743), 60d9a44245624a7682ec4ae233c12b99 (127.1.124.131:36381)
I20260808 13:06:46.104084 22917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:46.096911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:55495 (local address 127.1.124.129:36743)
0808 13:06:46.097136 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:46.097141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:46.097710 (+ 569us) server_negotiation.cc:407] Connection header received
0808 13:06:46.097969 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:46.097994 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:46.098097 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:46.098165 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:06:46.099099 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:46.100040 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:46.102123 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:46.103048 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:46.103173 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:46.103226 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:46.103645 (+ 419us) server_negotiation.cc:299] Negotiation successful
0808 13:06:46.103781 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 13:06:46.104274 22920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:46.098156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:51305 (local address 127.1.124.131:36381)
0808 13:06:46.098402 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:46.098408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:46.098472 (+ 64us) server_negotiation.cc:407] Connection header received
0808 13:06:46.098684 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:46.098689 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:46.098770 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:46.098893 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:06:46.099591 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:46.100520 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:46.102337 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:46.103170 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:46.103344 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:46.103420 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:46.103866 (+ 446us) server_negotiation.cc:299] Negotiation successful
0808 13:06:46.104008 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260808 13:06:46.104291 22937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:46.096812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:36743 (local address 127.1.124.130:55495)
0808 13:06:46.097582 (+ 770us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:46.097604 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:46.097755 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:46.098415 (+ 660us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:46.098422 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:46.098458 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:06:46.098872 (+ 414us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:46.098888 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:46.100233 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:46.100238 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:46.101950 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:46.101963 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:46.103276 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:46.103282 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:46.103482 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:46.103508 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 13:06:46.103591 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 13:06:46.103738 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"mutex_wait_us":71,"thread_start_us":427,"threads_started":1}
I20260808 13:06:46.104408 22938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:46.097736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:36381 (local address 127.1.124.130:51305)
0808 13:06:46.098385 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:46.098401 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:46.098535 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:46.098971 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:46.098979 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:46.099008 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:06:46.099400 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:46.099412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:46.100672 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:46.100676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:46.102175 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:46.102189 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:46.103404 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:46.103408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:46.103679 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:46.103712 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 13:06:46.103802 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 13:06:46.103926 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"mutex_wait_us":2,"thread_start_us":313,"threads_started":1}
I20260808 13:06:46.105407 22815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3787f2dd4574dd29a66c75f38b3c81b" candidate_uuid: "0ed0b11d02ae46c1ab5bdd11adc6621b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60d9a44245624a7682ec4ae233c12b99" is_pre_election: true
I20260808 13:06:46.105432 22414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3787f2dd4574dd29a66c75f38b3c81b" candidate_uuid: "0ed0b11d02ae46c1ab5bdd11adc6621b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37e05c9fa08490c8520dd4ffc2a83cf" is_pre_election: true
I20260808 13:06:46.105942 22815 raft_consensus.cc:2454] T a3787f2dd4574dd29a66c75f38b3c81b P 60d9a44245624a7682ec4ae233c12b99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ed0b11d02ae46c1ab5bdd11adc6621b in term 0.
I20260808 13:06:46.105947 22414 raft_consensus.cc:2454] T a3787f2dd4574dd29a66c75f38b3c81b P c37e05c9fa08490c8520dd4ffc2a83cf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ed0b11d02ae46c1ab5bdd11adc6621b in term 0.
I20260808 13:06:46.106608 22524 leader_election.cc:302] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ed0b11d02ae46c1ab5bdd11adc6621b, c37e05c9fa08490c8520dd4ffc2a83cf; no voters:
I20260808 13:06:46.107010 22931 raft_consensus.cc:2781] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:06:46.107195 22931 raft_consensus.cc:482] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:06:46.107336 22931 raft_consensus.cc:3037] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:06:46.109858 22931 raft_consensus.cc:504] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37e05c9fa08490c8520dd4ffc2a83cf" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 36743 } } peers { permanent_uuid: "0ed0b11d02ae46c1ab5bdd11adc6621b" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37723 } } peers { permanent_uuid: "60d9a44245624a7682ec4ae233c12b99" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 36381 } }
I20260808 13:06:46.110632 22931 leader_election.cc:288] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b [CANDIDATE]: Term 1 election: Requested vote from peers c37e05c9fa08490c8520dd4ffc2a83cf (127.1.124.129:36743), 60d9a44245624a7682ec4ae233c12b99 (127.1.124.131:36381)
I20260808 13:06:46.111223 22414 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3787f2dd4574dd29a66c75f38b3c81b" candidate_uuid: "0ed0b11d02ae46c1ab5bdd11adc6621b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c37e05c9fa08490c8520dd4ffc2a83cf"
I20260808 13:06:46.111281 22815 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3787f2dd4574dd29a66c75f38b3c81b" candidate_uuid: "0ed0b11d02ae46c1ab5bdd11adc6621b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "60d9a44245624a7682ec4ae233c12b99"
I20260808 13:06:46.111495 22414 raft_consensus.cc:3037] T a3787f2dd4574dd29a66c75f38b3c81b P c37e05c9fa08490c8520dd4ffc2a83cf [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:06:46.111547 22815 raft_consensus.cc:3037] T a3787f2dd4574dd29a66c75f38b3c81b P 60d9a44245624a7682ec4ae233c12b99 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:06:46.113967 22414 raft_consensus.cc:2454] T a3787f2dd4574dd29a66c75f38b3c81b P c37e05c9fa08490c8520dd4ffc2a83cf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ed0b11d02ae46c1ab5bdd11adc6621b in term 1.
I20260808 13:06:46.113965 22815 raft_consensus.cc:2454] T a3787f2dd4574dd29a66c75f38b3c81b P 60d9a44245624a7682ec4ae233c12b99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ed0b11d02ae46c1ab5bdd11adc6621b in term 1.
I20260808 13:06:46.114447 22524 leader_election.cc:302] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ed0b11d02ae46c1ab5bdd11adc6621b, 60d9a44245624a7682ec4ae233c12b99; no voters:
I20260808 13:06:46.114784 22931 raft_consensus.cc:2781] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:06:46.115309 22931 raft_consensus.cc:686] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b [term 1 LEADER]: Becoming Leader. State: Replica: 0ed0b11d02ae46c1ab5bdd11adc6621b, State: Running, Role: LEADER
I20260808 13:06:46.115815 22931 consensus_queue.cc:237] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37e05c9fa08490c8520dd4ffc2a83cf" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 36743 } } peers { permanent_uuid: "0ed0b11d02ae46c1ab5bdd11adc6621b" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37723 } } peers { permanent_uuid: "60d9a44245624a7682ec4ae233c12b99" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 36381 } }
W20260808 13:06:46.116598 22886 tablet.cc:2369] T a3787f2dd4574dd29a66c75f38b3c81b P 60d9a44245624a7682ec4ae233c12b99: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:06:46.121078 22166 catalog_manager.cc:5495] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ed0b11d02ae46c1ab5bdd11adc6621b (127.1.124.130). New cstate: current_term: 1 leader_uuid: "0ed0b11d02ae46c1ab5bdd11adc6621b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c37e05c9fa08490c8520dd4ffc2a83cf" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 36743 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ed0b11d02ae46c1ab5bdd11adc6621b" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "60d9a44245624a7682ec4ae233c12b99" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 36381 } health_report { overall_health: UNKNOWN } } }
I20260808 13:06:46.162663 22919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:46.157098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47090 (local address 127.1.124.130:37723)
0808 13:06:46.157338 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:46.157346 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:46.157367 (+ 21us) server_negotiation.cc:407] Connection header received
0808 13:06:46.157475 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:46.157479 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:46.157555 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:46.157728 (+ 173us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:06:46.158122 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:46.158912 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:46.160392 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:46.160835 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:46.160977 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:46.161509 (+ 532us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:06:46.162022 (+ 513us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:06:46.162218 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:46.162305 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 13:06:46.162388 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 13:06:46.171417 22815 raft_consensus.cc:1261] T a3787f2dd4574dd29a66c75f38b3c81b P 60d9a44245624a7682ec4ae233c12b99 [term 1 FOLLOWER]: Refusing update from remote peer 0ed0b11d02ae46c1ab5bdd11adc6621b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:06:46.171440 22414 raft_consensus.cc:1261] T a3787f2dd4574dd29a66c75f38b3c81b P c37e05c9fa08490c8520dd4ffc2a83cf [term 1 FOLLOWER]: Refusing update from remote peer 0ed0b11d02ae46c1ab5bdd11adc6621b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:06:46.172435 22939 consensus_queue.cc:1035] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b [LEADER]: Connected to new peer: Peer: permanent_uuid: "c37e05c9fa08490c8520dd4ffc2a83cf" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 36743 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:06:46.172894 22931 consensus_queue.cc:1035] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b [LEADER]: Connected to new peer: Peer: permanent_uuid: "60d9a44245624a7682ec4ae233c12b99" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 36381 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:06:46.200783 22168 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:06:46.201123 22168 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:06:46.205327 22168 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7a9fdf4b0ef141e1b0dc41b2d37ee916: Sending DeleteTablet for 3 replicas of tablet a3787f2dd4574dd29a66c75f38b3c81b
I20260808 13:06:46.206600 22383 tablet_service.cc:1514] Processing DeleteTablet for tablet a3787f2dd4574dd29a66c75f38b3c81b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:06:46 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:39298
I20260808 13:06:46.206748 22785 tablet_service.cc:1514] Processing DeleteTablet for tablet a3787f2dd4574dd29a66c75f38b3c81b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:06:46 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:52644
I20260808 13:06:46.211318 22585 tablet_service.cc:1514] Processing DeleteTablet for tablet a3787f2dd4574dd29a66c75f38b3c81b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:06:46 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:47074
I20260808 13:06:46.213558 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 22233
I20260808 13:06:46.217473 22958 tablet_replica.cc:330] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b: stopping tablet replica
I20260808 13:06:46.218703 22958 raft_consensus.cc:2229] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:06:46.221027 22958 raft_consensus.cc:2258] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:06:46.222776 22955 tablet_replica.cc:330] T a3787f2dd4574dd29a66c75f38b3c81b P c37e05c9fa08490c8520dd4ffc2a83cf: stopping tablet replica
I20260808 13:06:46.223491 22957 tablet_replica.cc:330] T a3787f2dd4574dd29a66c75f38b3c81b P 60d9a44245624a7682ec4ae233c12b99: stopping tablet replica
I20260808 13:06:46.224342 22955 raft_consensus.cc:2229] T a3787f2dd4574dd29a66c75f38b3c81b P c37e05c9fa08490c8520dd4ffc2a83cf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:06:46.225069 22955 raft_consensus.cc:2258] T a3787f2dd4574dd29a66c75f38b3c81b P c37e05c9fa08490c8520dd4ffc2a83cf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:06:46.225703 22957 raft_consensus.cc:2229] T a3787f2dd4574dd29a66c75f38b3c81b P 60d9a44245624a7682ec4ae233c12b99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:06:46.226075 22958 ts_tablet_manager.cc:1905] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:06:46.226305 22957 raft_consensus.cc:2258] T a3787f2dd4574dd29a66c75f38b3c81b P 60d9a44245624a7682ec4ae233c12b99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:06:46.229914 22957 ts_tablet_manager.cc:1905] T a3787f2dd4574dd29a66c75f38b3c81b P 60d9a44245624a7682ec4ae233c12b99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:06:46.233443 22955 ts_tablet_manager.cc:1905] T a3787f2dd4574dd29a66c75f38b3c81b P c37e05c9fa08490c8520dd4ffc2a83cf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:06:46.234226 22958 ts_tablet_manager.cc:1918] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:06:46.234508 22958 log.cc:1196] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/a3787f2dd4574dd29a66c75f38b3c81b
I20260808 13:06:46.235193 22958 ts_tablet_manager.cc:1939] T a3787f2dd4574dd29a66c75f38b3c81b P 0ed0b11d02ae46c1ab5bdd11adc6621b: Deleting consensus metadata
I20260808 13:06:46.237305 22143 catalog_manager.cc:4841] TS 0ed0b11d02ae46c1ab5bdd11adc6621b (127.1.124.130:37723): tablet a3787f2dd4574dd29a66c75f38b3c81b (table test-table [id=97fef078284742fdb778924793e7bed8]) successfully deleted
I20260808 13:06:46.239800 22917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:46.215496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39300 (local address 127.1.124.129:36743)
0808 13:06:46.215676 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:46.215682 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:46.215701 (+ 19us) server_negotiation.cc:407] Connection header received
0808 13:06:46.215862 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:46.215869 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:46.215955 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:46.216052 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:46.223871 (+ 7819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:46.225159 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:46.227399 (+ 2240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:46.228042 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:46.228237 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:46.231871 (+ 3634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:46.231885 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:46.231903 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:46.231967 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:46.236228 (+ 4261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:46.237167 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:46.237177 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:46.237180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:46.237357 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:46.237842 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:46.237847 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:46.237850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:46.238334 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:46.238644 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:46.239106 (+ 462us) server_negotiation.cc:299] Negotiation successful
0808 13:06:46.239368 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 13:06:46.240518 22462 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:06:46.244624 22957 ts_tablet_manager.cc:1918] T a3787f2dd4574dd29a66c75f38b3c81b P 60d9a44245624a7682ec4ae233c12b99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:06:46.244817 22957 log.cc:1196] T a3787f2dd4574dd29a66c75f38b3c81b P 60d9a44245624a7682ec4ae233c12b99: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/a3787f2dd4574dd29a66c75f38b3c81b
I20260808 13:06:46.245440 22957 ts_tablet_manager.cc:1939] T a3787f2dd4574dd29a66c75f38b3c81b P 60d9a44245624a7682ec4ae233c12b99: Deleting consensus metadata
I20260808 13:06:46.247491 22143 catalog_manager.cc:4841] TS 60d9a44245624a7682ec4ae233c12b99 (127.1.124.131:36381): tablet a3787f2dd4574dd29a66c75f38b3c81b (table test-table [id=97fef078284742fdb778924793e7bed8]) successfully deleted
I20260808 13:06:46.361445 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 22233
I20260808 13:06:46.381894 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 22481
W20260808 13:06:46.381889 22143 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.124.129:36743 (error 108)
I20260808 13:06:46.384178 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:46.383608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:36743 (local address 127.0.0.1:39308)
0808 13:06:46.383818 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:46.383957 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:36743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260808 13:06:46.384444 22143 catalog_manager.cc:4584] TS c37e05c9fa08490c8520dd4ffc2a83cf (127.1.124.129:36743): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a3787f2dd4574dd29a66c75f38b3c81b: Network error: Client connection negotiation failed: client connection to 127.1.124.129:36743: connect: Connection refused (error 111)
I20260808 13:06:46.397732 22919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:46.382909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47096 (local address 127.1.124.130:37723)
0808 13:06:46.383119 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:46.383127 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:46.383150 (+ 23us) server_negotiation.cc:407] Connection header received
0808 13:06:46.383789 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:46.383793 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:46.383905 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:46.384043 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:46.385616 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:46.386478 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:46.387790 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:46.388375 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:46.388524 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:46.391222 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:46.391236 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:46.391251 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:46.391296 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:46.394411 (+ 3115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:46.395131 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:46.395138 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:46.395141 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:46.395292 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:46.395793 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:46.395798 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:46.395802 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:46.396446 (+ 644us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:46.396813 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:46.397081 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 13:06:46.397330 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 13:06:46.398590 22676 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:06:46.438534 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:46.438055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:36743 (local address 127.0.0.1:39318)
0808 13:06:46.438270 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:46.438385 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:36743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260808 13:06:46.439644 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:46.439279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:36743 (local address 127.0.0.1:39330)
0808 13:06:46.439430 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:46.439514 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:36743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 13:06:46.500258 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:46.499714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:36743 (local address 127.0.0.1:39346)
0808 13:06:46.499938 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:46.500041 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:36743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 13:06:46.501407 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:46.501034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:36743 (local address 127.0.0.1:39358)
0808 13:06:46.501213 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:46.501292 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:36743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 13:06:46.514940 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 22481
I20260808 13:06:46.540194 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 22688
I20260808 13:06:46.554041 22920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:46.541126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52656 (local address 127.1.124.131:36381)
0808 13:06:46.541336 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:46.541341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:46.541362 (+ 21us) server_negotiation.cc:407] Connection header received
0808 13:06:46.541534 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:46.541546 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:46.541616 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:46.541714 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:46.543195 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:46.543927 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:46.545340 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:46.545748 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:46.545849 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:46.548345 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:46.548358 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:46.548374 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:46.548411 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:46.551266 (+ 2855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:46.551898 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:46.551905 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:46.551907 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:46.552039 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:46.552577 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:46.552580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:46.552583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:46.552922 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:46.553205 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:46.553461 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 13:06:46.553642 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 13:06:46.554728 22879 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:06:46.582258 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:46.581687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:36743 (local address 127.0.0.1:39364)
0808 13:06:46.581928 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:46.582052 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:36743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260808 13:06:46.583356 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:46.582982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:36743 (local address 127.0.0.1:39368)
0808 13:06:46.583179 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:46.583249 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:36743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260808 13:06:46.646695 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 22688
I20260808 13:06:46.671115 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 22117
I20260808 13:06:46.672483 22224 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:06:46.791543 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 22117
I20260808 13:06:46.791671 22921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:46.791113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:36743 (local address 127.0.0.1:39382)
0808 13:06:46.791339 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:46.791439 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:36743: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 13:06:46.828773 1522 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 13:06:47.171821 1522 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 13:06:47.172145 1522 mini_ranger.cc:66] Stopping Ranger...
I20260808 13:06:47.519351 1522 mini_ranger.cc:68] Stopped Ranger
2026-08-08 13:06:47.519 UTC [20667] LOG: received smart shutdown request
2026-08-08 13:06:47.523 UTC [20667] LOG: background worker "logical replication launcher" (PID 20678) exited with exit code 1
2026-08-08 13:06:47.523 UTC [20673] LOG: shutting down
2026-08-08 13:06:47.695 UTC [20667] LOG: database system is shut down
2026-08-08T13:06:47Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (55366 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:06:47 dist-test-slave-t3q3 krb5kdc[22979](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:06:47 dist-test-slave-t3q3 krb5kdc[22979](info): set up 2 sockets
Aug 08 13:06:47 dist-test-slave-t3q3 krb5kdc[22979](info): commencing operation
krb5kdc: starting...
W20260808 13:06:49.173384 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:06:49 dist-test-slave-t3q3 krb5kdc[22979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:06:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:06:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:06:49.476167 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:40701
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:35439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:40701,127.1.124.189:34705,127.1.124.188:34251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:06:49.834262 22997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:06:49.834580 22997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:06:49.834802 22997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:06:49.842319 22997 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:06:49.842448 22997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:06:49.842602 22997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:06:49.842707 22997 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:06:49.842797 22997 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:06:49.852900 22997 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:35439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:40701,127.1.124.189:34705,127.1.124.188:34251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:40701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:06:49.854383 22997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:06:49.855789 22997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:06:49.868358 23005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:49.868459 23007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:49.868398 23010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:06:49.869788 22997 server_base.cc:1034] running on GCE node
I20260808 13:06:49.870718 22997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:06:49.872643 22997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:06:49.874056 22997 hybrid_clock.cc:648] HybridClock initialized: now 1786194409874026 us; error 47 us; skew 500 ppm
I20260808 13:06:49.874464 22997 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:06:49 dist-test-slave-t3q3 krb5kdc[22979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:06:49.877875 22997 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:06:49.879637 22997 webserver.cc:460] Webserver started at http://127.1.124.190:33563/ using document root <none> and password file <none>
I20260808 13:06:49.880474 22997 fs_manager.cc:359] Metadata directory not provided
I20260808 13:06:49.880635 22997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:06:49.880975 22997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:06:49.884230 22997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "2bef1e6073e04d5488c80f87e11020c7"
format_stamp: "Formatted at 2026-08-08 13:06:49 on dist-test-slave-t3q3"
server_key: "c4aa8ce6c828f33ad97b49d8b47772c7"
server_key_iv: "057e48bdec87919030dacd35a1233b36"
server_key_version: "encryptionkey@0"
I20260808 13:06:49.885103 22997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "2bef1e6073e04d5488c80f87e11020c7"
format_stamp: "Formatted at 2026-08-08 13:06:49 on dist-test-slave-t3q3"
server_key: "c4aa8ce6c828f33ad97b49d8b47772c7"
server_key_iv: "057e48bdec87919030dacd35a1233b36"
server_key_version: "encryptionkey@0"
I20260808 13:06:49.890748 22997 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 13:06:49.894511 23019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:49.895964 22997 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:06:49.896209 22997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "2bef1e6073e04d5488c80f87e11020c7"
format_stamp: "Formatted at 2026-08-08 13:06:49 on dist-test-slave-t3q3"
server_key: "c4aa8ce6c828f33ad97b49d8b47772c7"
server_key_iv: "057e48bdec87919030dacd35a1233b36"
server_key_version: "encryptionkey@0"
I20260808 13:06:49.896421 22997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:06:49.928560 22997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:06:49.933557 22997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:06:49.933879 22997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:06:49.953472 22997 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:40701
I20260808 13:06:49.953507 23101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:40701 every 8 connection(s)
I20260808 13:06:49.955027 22997 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:06:49.956310 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 22997
I20260808 13:06:49.956583 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:06:49.957326 1522 external_mini_cluster.cc:1428] Setting key ee80a6cce202d910f35163f29e5d58ed
I20260808 13:06:49.959214 23102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:06:49.966017 23102 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:06:49 dist-test-slave-t3q3 krb5kdc[22979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.124.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.189@KRBTEST.COM" created.
I20260808 13:06:49.990561 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:49.969158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:40701 (local address 127.0.0.1:56106)
0808 13:06:49.970045 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:49.970103 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:49.972310 (+ 2207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:49.973309 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:49.973314 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:49.973874 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:06:49.974206 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:49.974223 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:49.976686 (+ 2463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:49.976690 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:49.977797 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:49.977810 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:49.979227 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:49.979232 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:49.979364 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:49.980112 (+ 748us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:06:49.980148 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:06:49.982371 (+ 2223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:06:49.986392 (+ 4021us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:49.986404 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:49.986414 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:49.986841 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:49.987256 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:49.987259 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:49.987262 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:49.987415 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:49.988566 (+ 1151us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:06:49.988577 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:06:49.988810 (+ 233us) client_negotiation.cc:769] Sending connection context
0808 13:06:49.989035 (+ 225us) client_negotiation.cc:240] Negotiation successful
0808 13:06:49.989462 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":387,"threads_started":1}
I20260808 13:06:49.990604 23106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:49.970517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56106 (local address 127.1.124.190:40701)
0808 13:06:49.971284 (+ 767us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:49.971295 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:49.971351 (+ 56us) server_negotiation.cc:407] Connection header received
0808 13:06:49.972698 (+ 1347us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:49.972708 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:49.973048 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:49.973276 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:49.974425 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:49.976503 (+ 2078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:49.977950 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:49.978983 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:49.979187 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:49.982646 (+ 3459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:49.982658 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:49.982677 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:49.982716 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:49.986254 (+ 3538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:49.986959 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:49.986966 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:49.986968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:49.987119 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:49.987553 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:49.987557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:49.987560 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:49.988411 (+ 851us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:49.988615 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:49.989242 (+ 627us) server_negotiation.cc:299] Negotiation successful
0808 13:06:49.989692 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":97,"server-negotiator.queue_time_us":488,"thread_start_us":316,"threads_started":1}
I20260808 13:06:49.994277 23102 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:06:49.995507 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:49.995057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:34705 (local address 127.0.0.1:41486)
0808 13:06:49.995262 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:49.995386 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
W20260808 13:06:49.995826 23022 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.124.189:34705: connect: Connection refused (error 111)
I20260808 13:06:49.997548 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:49.997160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:34705 (local address 127.0.0.1:41496)
0808 13:06:49.997321 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:49.997408 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 13:06:49.997772 23102 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.189:34705: Network error: Client connection negotiation failed: client connection to 127.1.124.189:34705: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.189.keytab.
Entry for principal kudu/127.1.124.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.189.keytab.
I20260808 13:06:50.025341 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.189:34705
--webserver_interface=127.1.124.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.189.keytab
--principal=kudu/127.1.124.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:35439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:40701,127.1.124.189:34705,127.1.124.188:34251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:06:50.047220 23102 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } attempt: 1
I20260808 13:06:50.048779 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.048040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:34705 (local address 127.0.0.1:41498)
0808 13:06:50.048381 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.048545 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159}
I20260808 13:06:50.049990 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.049511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:34705 (local address 127.0.0.1:41514)
0808 13:06:50.049701 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.049804 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260808 13:06:50.050259 23102 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.189:34705: Network error: Client connection negotiation failed: client connection to 127.1.124.189:34705: connect: Connection refused (error 111)
I20260808 13:06:50.118541 23102 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } attempt: 2
I20260808 13:06:50.119943 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.119267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:34705 (local address 127.0.0.1:41522)
0808 13:06:50.119596 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.119733 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260808 13:06:50.121184 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.120697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:34705 (local address 127.0.0.1:41538)
0808 13:06:50.120893 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.121008 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 13:06:50.121398 23102 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.189:34705: Network error: Client connection negotiation failed: client connection to 127.1.124.189:34705: connect: Connection refused (error 111)
I20260808 13:06:50.212707 23102 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } attempt: 3
I20260808 13:06:50.214190 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.213547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:34705 (local address 127.0.0.1:41548)
0808 13:06:50.213844 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.213984 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260808 13:06:50.215327 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.214886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:34705 (local address 127.0.0.1:41558)
0808 13:06:50.215051 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.215159 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260808 13:06:50.215561 23102 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.189:34705: Network error: Client connection negotiation failed: client connection to 127.1.124.189:34705: connect: Connection refused (error 111)
I20260808 13:06:50.358860 23102 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } attempt: 4
I20260808 13:06:50.360356 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.359629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:34705 (local address 127.0.0.1:41564)
0808 13:06:50.360004 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.360181 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186}
I20260808 13:06:50.361457 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.361030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:34705 (local address 127.0.0.1:41572)
0808 13:06:50.361199 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.361311 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:34705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260808 13:06:50.361689 23102 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.189:34705: Network error: Client connection negotiation failed: client connection to 127.1.124.189:34705: connect: Connection refused (error 111)
W20260808 13:06:50.381000 23110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:06:50.381295 23110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:06:50.381388 23110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:06:50.388739 23110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:06:50.388847 23110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:06:50.388895 23110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:06:50.388953 23110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:06:50.388998 23110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:06:50.399308 23110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:35439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:40701,127.1.124.189:34705,127.1.124.188:34251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.189.keytab
--principal=kudu/127.1.124.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.189:34705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.124.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:06:50.400764 23110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:06:50.402107 23110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:06:50.414247 23121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:50.414288 23120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:50.414324 23123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:06:50.415128 23110 server_base.cc:1034] running on GCE node
I20260808 13:06:50.415954 23110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:06:50.417466 23110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:06:50.418684 23110 hybrid_clock.cc:648] HybridClock initialized: now 1786194410418634 us; error 46 us; skew 500 ppm
I20260808 13:06:50.419093 23110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:06:50 dist-test-slave-t3q3 krb5kdc[22979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194410, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:06:50.422483 23110 init.cc:377] Logged in from keytab as kudu/127.1.124.189@KRBTEST.COM (short username kudu)
I20260808 13:06:50.424135 23110 webserver.cc:460] Webserver started at http://127.1.124.189:36387/ using document root <none> and password file <none>
I20260808 13:06:50.424863 23110 fs_manager.cc:359] Metadata directory not provided
I20260808 13:06:50.425007 23110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:06:50.425350 23110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:06:50.428561 23110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-1/data/instance:
uuid: "74ec708a30da43c8b3461289d17f6593"
format_stamp: "Formatted at 2026-08-08 13:06:50 on dist-test-slave-t3q3"
server_key: "41bf1cfa944dd531f69a836149cdb241"
server_key_iv: "9f4ffaf1738130f2484ca8983f6c12ca"
server_key_version: "encryptionkey@0"
I20260808 13:06:50.429440 23110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-1/wal/instance:
uuid: "74ec708a30da43c8b3461289d17f6593"
format_stamp: "Formatted at 2026-08-08 13:06:50 on dist-test-slave-t3q3"
server_key: "41bf1cfa944dd531f69a836149cdb241"
server_key_iv: "9f4ffaf1738130f2484ca8983f6c12ca"
server_key_version: "encryptionkey@0"
I20260808 13:06:50.434944 23110 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 13:06:50.438584 23131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:50.440121 23110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:06:50.440322 23110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-1/wal
uuid: "74ec708a30da43c8b3461289d17f6593"
format_stamp: "Formatted at 2026-08-08 13:06:50 on dist-test-slave-t3q3"
server_key: "41bf1cfa944dd531f69a836149cdb241"
server_key_iv: "9f4ffaf1738130f2484ca8983f6c12ca"
server_key_version: "encryptionkey@0"
I20260808 13:06:50.440537 23110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:06:50.462554 23110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:06:50.467569 23110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:06:50.467922 23110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:06:50.487941 23110 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.189:34705
I20260808 13:06:50.487984 23203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.189:34705 every 8 connection(s)
I20260808 13:06:50.489389 23110 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-1/data/info.pb
I20260808 13:06:50.493261 23204 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:06:50.495440 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 23110
I20260808 13:06:50.495653 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-1/wal/instance
I20260808 13:06:50.496270 1522 external_mini_cluster.cc:1428] Setting key 6b9536d0be67ff1bdcb0a94b63e7986b
I20260808 13:06:50.502938 23204 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:06:50 dist-test-slave-t3q3 krb5kdc[22979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194410, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.189@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.124.188@KRBTEST.COM; defaulting to no policy
I20260808 13:06:50.525972 23208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.506518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56116 (local address 127.1.124.190:40701)
0808 13:06:50.507070 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:50.507077 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:50.508304 (+ 1227us) server_negotiation.cc:407] Connection header received
0808 13:06:50.510229 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:50.510235 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:50.510330 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:50.510458 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:50.511993 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:50.512790 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:50.514381 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:50.514800 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:50.514934 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:50.518875 (+ 3941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:50.518885 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:50.518893 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:50.518959 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:50.522800 (+ 3841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:50.523602 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:50.523611 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:50.523617 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:50.523763 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:50.524310 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:50.524316 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:50.524321 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:50.524605 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:50.524745 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:50.525534 (+ 789us) server_negotiation.cc:299] Negotiation successful
0808 13:06:50.525708 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":271,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.188@KRBTEST.COM" created.
I20260808 13:06:50.527185 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.507173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:40701 (local address 127.0.0.1:56116)
0808 13:06:50.508051 (+ 878us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.508149 (+ 98us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:50.509939 (+ 1790us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:50.510722 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:50.510731 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:50.511251 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:06:50.511814 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:50.511832 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:50.512979 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:50.512983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:50.514222 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:50.514234 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:50.515035 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:50.515043 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:50.515284 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:50.516001 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:06:50.516024 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:06:50.518605 (+ 2581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:06:50.522964 (+ 4359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:50.522971 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:50.522986 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:50.523468 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:50.523875 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:50.523878 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:50.523880 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:50.524189 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:50.524725 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:06:50.524731 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:06:50.525171 (+ 440us) client_negotiation.cc:769] Sending connection context
0808 13:06:50.525432 (+ 261us) client_negotiation.cc:240] Negotiation successful
0808 13:06:50.525864 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"thread_start_us":329,"threads_started":1}
I20260808 13:06:50.529835 23204 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:06:50 dist-test-slave-t3q3 krb5kdc[22979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194410, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.189@KRBTEST.COM for kudu/127.1.124.189@KRBTEST.COM
I20260808 13:06:50.545600 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.530643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:34705 (local address 127.0.0.1:41584)
0808 13:06:50.530892 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.530915 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:50.531026 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:50.532032 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:50.532043 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:50.532602 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:06:50.532823 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:50.532837 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:50.534188 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:50.534195 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:50.535695 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:50.535712 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:50.536609 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:50.536617 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:50.536860 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:50.537461 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:06:50.537488 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:06:50.539665 (+ 2177us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:06:50.542872 (+ 3207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:50.542880 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:50.542884 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:50.543253 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:50.543714 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:50.543720 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:50.543723 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:50.543909 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:50.544775 (+ 866us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:06:50.544781 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:06:50.544926 (+ 145us) client_negotiation.cc:769] Sending connection context
0808 13:06:50.545121 (+ 195us) client_negotiation.cc:240] Negotiation successful
0808 13:06:50.545345 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260808 13:06:50.546108 23214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.530899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41584 (local address 127.1.124.189:34705)
0808 13:06:50.531590 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:50.531595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:50.531621 (+ 26us) server_negotiation.cc:407] Connection header received
0808 13:06:50.531662 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:50.531668 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:50.531843 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:50.531952 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:50.532994 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:50.534043 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:50.535834 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:50.536456 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:50.536552 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:50.539889 (+ 3337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:50.539900 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:50.539913 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:50.539945 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:50.542741 (+ 2796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:50.543402 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:50.543409 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:50.543411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:50.543576 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:50.544046 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:50.544054 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:50.544081 (+ 27us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:50.544631 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:50.544887 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:50.545303 (+ 416us) server_negotiation.cc:299] Negotiation successful
0808 13:06:50.545610 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":391,"threads_started":1}
I20260808 13:06:50.548503 23204 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:06:50.549604 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.549194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:34251 (local address 127.0.0.1:59046)
0808 13:06:50.549366 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.549470 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 13:06:50.549907 23132 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
I20260808 13:06:50.551486 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.551182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:34251 (local address 127.0.0.1:59056)
0808 13:06:50.551319 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.551395 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 13:06:50.551790 23204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:34251: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.188.keytab.
Entry for principal kudu/127.1.124.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.188.keytab.
I20260808 13:06:50.563547 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.188:34251
--webserver_interface=127.1.124.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.188.keytab
--principal=kudu/127.1.124.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:35439
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:40701,127.1.124.189:34705,127.1.124.188:34251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:06:50.601135 23204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } attempt: 1
I20260808 13:06:50.602397 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.601914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:34251 (local address 127.0.0.1:59068)
0808 13:06:50.602114 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.602240 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260808 13:06:50.603500 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.603167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:34251 (local address 127.0.0.1:59084)
0808 13:06:50.603325 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.603395 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 13:06:50.603754 23204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:34251: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
I20260808 13:06:50.660987 23102 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } attempt: 5
Aug 08 13:06:50 dist-test-slave-t3q3 krb5kdc[22979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for kudu/127.1.124.189@KRBTEST.COM
I20260808 13:06:50.672019 23204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } attempt: 2
I20260808 13:06:50.673102 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.672658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:34251 (local address 127.0.0.1:59092)
0808 13:06:50.672854 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.672943 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260808 13:06:50.674278 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.673887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:34251 (local address 127.0.0.1:59094)
0808 13:06:50.674068 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.674143 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260808 13:06:50.674477 23204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:34251: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
I20260808 13:06:50.675396 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.661849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:34705 (local address 127.0.0.1:41588)
0808 13:06:50.662092 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.662115 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:50.662258 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:50.662669 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:50.662683 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:50.663088 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:06:50.663281 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:50.663290 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:50.664391 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:50.664399 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:50.665278 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:50.665289 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:50.666062 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:50.666067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:50.666286 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:50.666937 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:06:50.666955 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:06:50.669333 (+ 2378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:06:50.672660 (+ 3327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:50.672666 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:50.672670 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:50.673219 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:50.673638 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:50.673643 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:50.673647 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:50.673834 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:50.674534 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:06:50.674541 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:06:50.674733 (+ 192us) client_negotiation.cc:769] Sending connection context
0808 13:06:50.674883 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 13:06:50.675125 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260808 13:06:50.675596 23214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.661975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41588 (local address 127.1.124.189:34705)
0808 13:06:50.662163 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:50.662170 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:50.662214 (+ 44us) server_negotiation.cc:407] Connection header received
0808 13:06:50.662414 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:50.662427 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:50.662509 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:50.662624 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:50.663465 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:50.664257 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:50.665430 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:50.665906 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:50.666047 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:50.669548 (+ 3501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:50.669554 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:50.669557 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:50.669591 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:50.672490 (+ 2899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:50.673345 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:50.673351 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:50.673354 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:50.673501 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:50.673967 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:50.673973 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:50.673976 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:50.674398 (+ 422us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:50.674573 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:50.675021 (+ 448us) server_negotiation.cc:299] Negotiation successful
0808 13:06:50.675271 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 13:06:50.676566 23102 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:06:50.677772 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.677309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:34251 (local address 127.0.0.1:59106)
0808 13:06:50.677500 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.677597 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260808 13:06:50.678907 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.678493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:34251 (local address 127.0.0.1:59112)
0808 13:06:50.678657 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.678752 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260808 13:06:50.679143 23102 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:34251: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
I20260808 13:06:50.730403 23102 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } attempt: 1
I20260808 13:06:50.732456 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.731449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:34251 (local address 127.0.0.1:59120)
0808 13:06:50.731826 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.732048 (+ 222us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260808 13:06:50.734221 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.733793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:34251 (local address 127.0.0.1:59130)
0808 13:06:50.733964 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.734054 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 13:06:50.734673 23102 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:34251: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
I20260808 13:06:50.765710 23204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } attempt: 3
I20260808 13:06:50.767060 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.766554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:34251 (local address 127.0.0.1:59144)
0808 13:06:50.766787 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.766889 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260808 13:06:50.768343 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.767786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:34251 (local address 127.0.0.1:59148)
0808 13:06:50.768038 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.768214 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136}
W20260808 13:06:50.768565 23204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:34251: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
I20260808 13:06:50.802927 23102 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } attempt: 2
I20260808 13:06:50.804458 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.803717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:34251 (local address 127.0.0.1:59150)
0808 13:06:50.804035 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.804199 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260808 13:06:50.805794 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.805247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:34251 (local address 127.0.0.1:59162)
0808 13:06:50.805460 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.805578 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260808 13:06:50.806071 23102 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:34251: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
I20260808 13:06:50.911826 23204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } attempt: 4
I20260808 13:06:50.912321 23102 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } attempt: 3
I20260808 13:06:50.913518 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.912701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:34251 (local address 127.0.0.1:59168)
0808 13:06:50.913132 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.913258 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201}
I20260808 13:06:50.913553 23105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.912992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:34251 (local address 127.0.0.1:59178)
0808 13:06:50.913258 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.913351 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
I20260808 13:06:50.914984 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.914333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:34251 (local address 127.0.0.1:59194)
0808 13:06:50.914727 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.914827 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"threads_started":1}
W20260808 13:06:50.915364 23204 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:34251: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
I20260808 13:06:50.916759 23218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:50.914261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:34251 (local address 127.0.0.1:59180)
0808 13:06:50.916202 (+ 1941us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:50.916420 (+ 218us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1777,"thread_start_us":1501,"threads_started":1}
W20260808 13:06:50.917076 23102 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:34251: Network error: Client connection negotiation failed: client connection to 127.1.124.188:34251: connect: Connection refused (error 111)
W20260808 13:06:50.927716 23217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:06:50.928099 23217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:06:50.928264 23217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:06:50.936918 23217 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:06:50.937068 23217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:06:50.937167 23217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:06:50.937270 23217 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:06:50.937355 23217 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:06:50.947634 23217 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:35439
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:40701,127.1.124.189:34705,127.1.124.188:34251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.188.keytab
--principal=kudu/127.1.124.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.188:34251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.124.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:06:50.949156 23217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:06:50.950507 23217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:06:50.967038 23224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:50.967728 23225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:50.969584 23229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:06:50.970472 23217 server_base.cc:1034] running on GCE node
I20260808 13:06:50.971593 23217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:06:50.973691 23217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:06:50.974956 23217 hybrid_clock.cc:648] HybridClock initialized: now 1786194410974864 us; error 90 us; skew 500 ppm
I20260808 13:06:50.975456 23217 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:06:50 dist-test-slave-t3q3 krb5kdc[22979](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194410, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:06:50.979802 23217 init.cc:377] Logged in from keytab as kudu/127.1.124.188@KRBTEST.COM (short username kudu)
I20260808 13:06:50.981583 23217 webserver.cc:460] Webserver started at http://127.1.124.188:46085/ using document root <none> and password file <none>
I20260808 13:06:50.982364 23217 fs_manager.cc:359] Metadata directory not provided
I20260808 13:06:50.982499 23217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:06:50.982815 23217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:06:50.985823 23217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-2/data/instance:
uuid: "60f40066f7da4425a45c721fa1c4cf89"
format_stamp: "Formatted at 2026-08-08 13:06:50 on dist-test-slave-t3q3"
server_key: "d82256e3edbf77de163582a748fc2c0e"
server_key_iv: "aec062c27821aaff27922f1d97dc4594"
server_key_version: "encryptionkey@0"
I20260808 13:06:50.986647 23217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-2/wal/instance:
uuid: "60f40066f7da4425a45c721fa1c4cf89"
format_stamp: "Formatted at 2026-08-08 13:06:50 on dist-test-slave-t3q3"
server_key: "d82256e3edbf77de163582a748fc2c0e"
server_key_iv: "aec062c27821aaff27922f1d97dc4594"
server_key_version: "encryptionkey@0"
I20260808 13:06:50.992280 23217 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 13:06:50.995960 23236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:50.997506 23217 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:06:50.997684 23217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-2/wal
uuid: "60f40066f7da4425a45c721fa1c4cf89"
format_stamp: "Formatted at 2026-08-08 13:06:50 on dist-test-slave-t3q3"
server_key: "d82256e3edbf77de163582a748fc2c0e"
server_key_iv: "aec062c27821aaff27922f1d97dc4594"
server_key_version: "encryptionkey@0"
I20260808 13:06:50.997872 23217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:06:51.040720 23217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:06:51.045508 23217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:06:51.045815 23217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:06:51.065097 23217 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.188:34251
I20260808 13:06:51.065115 23313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.188:34251 every 8 connection(s)
I20260808 13:06:51.066556 23217 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-2/data/info.pb
I20260808 13:06:51.070333 23314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:06:51.073369 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 23217
I20260808 13:06:51.073581 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786194185613124-1522-0/minicluster-data/master-2/wal/instance
I20260808 13:06:51.074172 1522 external_mini_cluster.cc:1428] Setting key f2087cc9c7955df43c1fa88d62d60624
I20260808 13:06:51.078476 23314 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:06:51 dist-test-slave-t3q3 krb5kdc[22979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.188@KRBTEST.COM
Aug 08 13:06:51 dist-test-slave-t3q3 krb5kdc[22979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194410, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.188@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:06:51.094332 23102 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } attempt: 4
I20260808 13:06:51.098515 23320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:51.081608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56118 (local address 127.1.124.190:40701)
0808 13:06:51.082209 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:51.082215 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:51.082643 (+ 428us) server_negotiation.cc:407] Connection header received
0808 13:06:51.082819 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:51.082823 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:51.082884 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:51.082972 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:51.084424 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.085465 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.087347 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.087999 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.088176 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:51.091739 (+ 3563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:51.091745 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:51.091748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:51.091781 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:51.094491 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:51.095490 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:51.095494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:51.095497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:51.095664 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:51.096170 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:51.096186 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:51.096189 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:51.096565 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:51.096758 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:51.097649 (+ 891us) server_negotiation.cc:299] Negotiation successful
0808 13:06:51.097911 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":243,"threads_started":1}
I20260808 13:06:51.099464 23321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:51.081629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:40701 (local address 127.0.0.1:56118)
0808 13:06:51.082491 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:51.082533 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:51.082692 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:51.083075 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:51.083088 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:51.083917 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:06:51.084274 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:51.084284 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.085638 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.085642 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:51.087204 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:51.087215 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.088258 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.088263 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:51.088384 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:51.089002 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:06:51.089023 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:06:51.091541 (+ 2518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:06:51.094657 (+ 3116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:51.094668 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:51.094682 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:51.095325 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:51.095787 (+ 462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:51.095794 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:51.095800 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:51.096036 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:51.096728 (+ 692us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:06:51.096737 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:06:51.097151 (+ 414us) client_negotiation.cc:769] Sending connection context
0808 13:06:51.097479 (+ 328us) client_negotiation.cc:240] Negotiation successful
0808 13:06:51.098060 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"mutex_wait_us":3,"thread_start_us":462,"threads_started":1}
I20260808 13:06:51.099504 23317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:51.076224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59202 (local address 127.1.124.188:34251)
0808 13:06:51.077071 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:51.077083 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:51.077125 (+ 42us) server_negotiation.cc:407] Connection header received
0808 13:06:51.078750 (+ 1625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:51.078771 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:51.079155 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:51.079482 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:51.081493 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.083695 (+ 2202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.085624 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.086738 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.086906 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:51.091496 (+ 4590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:51.091509 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:51.091526 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:51.091556 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:51.094765 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:51.095675 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:51.095685 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:51.095688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:51.095897 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:51.096402 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:51.096406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:51.096409 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:51.097137 (+ 728us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:51.097466 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:51.098422 (+ 956us) server_negotiation.cc:299] Negotiation successful
0808 13:06:51.098709 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":641,"thread_start_us":431,"threads_started":1}
I20260808 13:06:51.101403 23314 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:06:51 dist-test-slave-t3q3 krb5kdc[22979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194409, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for kudu/127.1.124.188@KRBTEST.COM
Aug 08 13:06:51 dist-test-slave-t3q3 krb5kdc[22979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194410, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.188@KRBTEST.COM for kudu/127.1.124.189@KRBTEST.COM
Aug 08 13:06:51 dist-test-slave-t3q3 krb5kdc[22979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:06:51.113150 23322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:51.095033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59218 (local address 127.1.124.188:34251)
0808 13:06:51.095989 (+ 956us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:51.095999 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:51.096021 (+ 22us) server_negotiation.cc:407] Connection header received
0808 13:06:51.096085 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:51.096100 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:51.096183 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:51.096278 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:51.097551 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.098874 (+ 1323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.101704 (+ 2830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.102527 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.102660 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:51.106468 (+ 3808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:51.106473 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:51.106476 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:51.106506 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:51.110021 (+ 3515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:51.110810 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:51.110816 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:51.110821 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:51.110978 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:51.111477 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:51.111483 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:51.111488 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:51.111849 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:51.112011 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:51.112538 (+ 527us) server_negotiation.cc:299] Negotiation successful
0808 13:06:51.112749 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":808,"thread_start_us":589,"threads_started":1}
I20260808 13:06:51.113173 23218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:51.094940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:34251 (local address 127.0.0.1:59218)
0808 13:06:51.095222 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:51.095244 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:51.095356 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:51.096341 (+ 985us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:51.096346 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:51.096877 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:06:51.097298 (+ 421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:51.097314 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.099113 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.099119 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:51.100662 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:51.100689 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.102723 (+ 2034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.102728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:51.102881 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:51.103555 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:06:51.103579 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:06:51.106231 (+ 2652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:06:51.110175 (+ 3944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:51.110183 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:51.110188 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:51.110673 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:51.111171 (+ 498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:51.111177 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:51.111182 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:51.111346 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:51.111989 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:06:51.111996 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:06:51.112228 (+ 232us) client_negotiation.cc:769] Sending connection context
0808 13:06:51.112398 (+ 170us) client_negotiation.cc:240] Negotiation successful
0808 13:06:51.112690 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161}
I20260808 13:06:51.117763 23321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:51.102297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:34705 (local address 127.0.0.1:41598)
0808 13:06:51.102917 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:51.102935 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:51.103067 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:51.103631 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:51.103635 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:51.104147 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:06:51.104320 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:51.104330 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.105707 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.105712 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:51.107074 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:51.107090 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.108155 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.108161 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:51.108394 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:51.108956 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:06:51.108978 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:06:51.111679 (+ 2701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:06:51.115234 (+ 3555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:51.115239 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:51.115243 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:51.115608 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:51.115997 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:51.116003 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:51.116006 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:51.116210 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:51.116881 (+ 671us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:06:51.116884 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:06:51.117000 (+ 116us) client_negotiation.cc:769] Sending connection context
0808 13:06:51.117148 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 13:06:51.117399 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491}
I20260808 13:06:51.117854 23214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:51.102417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41598 (local address 127.1.124.189:34705)
0808 13:06:51.102582 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:51.102606 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:51.103024 (+ 418us) server_negotiation.cc:407] Connection header received
0808 13:06:51.103230 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:51.103236 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:51.103325 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:51.103404 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:51.104489 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.105564 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.107263 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.107983 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.108140 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:51.111870 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:51.111874 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:51.111878 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:51.111908 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:51.115094 (+ 3186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:51.115735 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:51.115747 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:51.115750 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:51.115882 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:51.116322 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:51.116328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:51.116331 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:51.116681 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:51.116827 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:51.117255 (+ 428us) server_negotiation.cc:299] Negotiation successful
0808 13:06:51.117485 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 13:06:51.119254 23314 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:06:51.120363 23102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7: Bootstrap starting.
I20260808 13:06:51.121445 23320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:51.103541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56120 (local address 127.1.124.190:40701)
0808 13:06:51.103690 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:51.103694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:51.103706 (+ 12us) server_negotiation.cc:407] Connection header received
0808 13:06:51.103741 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:51.103744 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:51.103816 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:51.103930 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:51.105514 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.106267 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.107850 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.108499 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.108645 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:51.114652 (+ 6007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:51.114658 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:51.114662 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:51.114695 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:51.118301 (+ 3606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:51.119162 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:51.119168 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:51.119172 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:51.119293 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:51.119686 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:51.119692 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:51.119695 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:51.120119 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:51.120407 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:51.120945 (+ 538us) server_negotiation.cc:299] Negotiation successful
0808 13:06:51.121156 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260808 13:06:51.124456 23102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7: Neither blocks nor log segments found. Creating new log.
I20260808 13:06:51.125737 23102 log.cc:824] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7: Log is configured to *not* fsync() on all Append() calls
I20260808 13:06:51.129380 23102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7: No bootstrap required, opened a new log
Aug 08 13:06:51 dist-test-slave-t3q3 krb5kdc[22979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194409, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.189@KRBTEST.COM
Aug 08 13:06:51 dist-test-slave-t3q3 krb5kdc[22979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194410, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.188@KRBTEST.COM for kudu/127.1.124.188@KRBTEST.COM
I20260808 13:06:51.139183 23214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:51.123072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41600 (local address 127.1.124.189:34705)
0808 13:06:51.123250 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:51.123258 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:51.123379 (+ 121us) server_negotiation.cc:407] Connection header received
0808 13:06:51.123529 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:51.123536 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:51.123612 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:51.123712 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:51.125012 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.126055 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.127885 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.128470 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.128611 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:51.133544 (+ 4933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:51.133553 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:51.133558 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:51.133595 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:51.136470 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:51.137105 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:51.137110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:51.137114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:51.137257 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:51.137645 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:51.137651 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:51.137655 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:51.137926 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:51.138232 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:51.138542 (+ 310us) server_negotiation.cc:299] Negotiation successful
0808 13:06:51.138784 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 13:06:51.140816 23321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:51.120007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:34251 (local address 127.0.0.1:59224)
0808 13:06:51.120218 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:51.120239 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:51.120329 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:51.126608 (+ 6279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:51.126612 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:51.127023 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:06:51.127155 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:51.127164 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.128267 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.128275 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:51.129573 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:51.129587 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.130616 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.130623 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:51.130857 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:51.131468 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:06:51.131494 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:06:51.134203 (+ 2709us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:06:51.138015 (+ 3812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:51.138023 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:51.138028 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:51.138507 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:51.138900 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:51.138904 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:51.138908 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:51.139048 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:51.139664 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:06:51.139670 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:06:51.139844 (+ 174us) client_negotiation.cc:769] Sending connection context
0808 13:06:51.140022 (+ 178us) client_negotiation.cc:240] Negotiation successful
0808 13:06:51.140395 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 13:06:51.140668 23102 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bef1e6073e04d5488c80f87e11020c7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 } } peers { permanent_uuid: "74ec708a30da43c8b3461289d17f6593" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } } peers { permanent_uuid: "60f40066f7da4425a45c721fa1c4cf89" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } }
I20260808 13:06:51.141130 23102 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:06:51.141279 23102 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bef1e6073e04d5488c80f87e11020c7, State: Initialized, Role: FOLLOWER
I20260808 13:06:51.141006 23322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:51.120000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59224 (local address 127.1.124.188:34251)
0808 13:06:51.126366 (+ 6366us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:51.126370 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:51.126383 (+ 13us) server_negotiation.cc:407] Connection header received
0808 13:06:51.126421 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:51.126424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:51.126483 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:51.126594 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:51.127304 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.128065 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.129739 (+ 1674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.130436 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.130580 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:51.134399 (+ 3819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:51.134405 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:51.134409 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:51.134442 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:51.137877 (+ 3435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:51.138638 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:51.138643 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:51.138645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:51.138781 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:51.139171 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:51.139177 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:51.139180 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:51.139513 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:51.139661 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:51.140191 (+ 530us) server_negotiation.cc:299] Negotiation successful
0808 13:06:51.140603 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":135,"server-negotiator.queue_time_us":6271}
I20260808 13:06:51.142001 23102 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bef1e6073e04d5488c80f87e11020c7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 } } peers { permanent_uuid: "74ec708a30da43c8b3461289d17f6593" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } } peers { permanent_uuid: "60f40066f7da4425a45c721fa1c4cf89" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } }
I20260808 13:06:51.144562 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:06:51.147137 23314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89: Bootstrap starting.
I20260808 13:06:51.151546 23314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89: Neither blocks nor log segments found. Creating new log.
I20260808 13:06:51.153373 23314 log.cc:824] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89: Log is configured to *not* fsync() on all Append() calls
I20260808 13:06:51.158294 23324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bef1e6073e04d5488c80f87e11020c7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 } } peers { permanent_uuid: "74ec708a30da43c8b3461289d17f6593" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } } peers { permanent_uuid: "60f40066f7da4425a45c721fa1c4cf89" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } } }
I20260808 13:06:51.160074 23102 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:06:51.160229 23324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:06:51.176433 23314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89: No bootstrap required, opened a new log
I20260808 13:06:51.180233 23214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:51.150939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41610 (local address 127.1.124.189:34705)
0808 13:06:51.157087 (+ 6148us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:51.157096 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:51.157788 (+ 692us) server_negotiation.cc:407] Connection header received
0808 13:06:51.160673 (+ 2885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:51.160680 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:51.160793 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:51.160888 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:51.164281 (+ 3393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.165778 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.168320 (+ 2542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.168995 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.169153 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:51.173177 (+ 4024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:51.173183 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:51.173188 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:51.173232 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:51.176871 (+ 3639us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:51.177740 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:51.177746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:51.177749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:51.177929 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:51.178410 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:51.178415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:51.178419 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:51.178796 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:51.179026 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:51.179572 (+ 546us) server_negotiation.cc:299] Negotiation successful
0808 13:06:51.179836 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5973}
I20260808 13:06:51.184434 23320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:51.150633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56128 (local address 127.1.124.190:40701)
0808 13:06:51.151677 (+ 1044us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:51.151683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:51.161280 (+ 9597us) server_negotiation.cc:407] Connection header received
0808 13:06:51.161832 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:51.161840 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:51.161935 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:51.162012 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:51.164965 (+ 2953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.166372 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.169314 (+ 2942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.170090 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.170232 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:51.175096 (+ 4864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:51.175106 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:51.175114 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:51.175170 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:51.179990 (+ 4820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:51.181110 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:51.181120 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:51.181126 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:51.181568 (+ 442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:51.182160 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:51.182168 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:51.182188 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:51.182645 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:51.183022 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:51.183738 (+ 716us) server_negotiation.cc:299] Negotiation successful
0808 13:06:51.184013 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":893}
W20260808 13:06:51.186919 23025 tablet.cc:2369] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:06:51.189280 23314 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bef1e6073e04d5488c80f87e11020c7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 } } peers { permanent_uuid: "74ec708a30da43c8b3461289d17f6593" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } } peers { permanent_uuid: "60f40066f7da4425a45c721fa1c4cf89" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } }
I20260808 13:06:51.189791 23314 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:06:51.189971 23314 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60f40066f7da4425a45c721fa1c4cf89, State: Initialized, Role: FOLLOWER
I20260808 13:06:51.190924 23314 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bef1e6073e04d5488c80f87e11020c7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 } } peers { permanent_uuid: "74ec708a30da43c8b3461289d17f6593" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } } peers { permanent_uuid: "60f40066f7da4425a45c721fa1c4cf89" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } }
I20260808 13:06:51.192163 23349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bef1e6073e04d5488c80f87e11020c7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 } } peers { permanent_uuid: "74ec708a30da43c8b3461289d17f6593" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } } peers { permanent_uuid: "60f40066f7da4425a45c721fa1c4cf89" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } } }
I20260808 13:06:51.192833 23349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:06:51.193940 23314 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:06:51.198184 23322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:51.155273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59236 (local address 127.1.124.188:34251)
0808 13:06:51.155486 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:51.155494 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:51.166089 (+ 10595us) server_negotiation.cc:407] Connection header received
0808 13:06:51.166576 (+ 487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:51.166582 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:51.166673 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:51.166745 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:51.172195 (+ 5450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.173264 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.184193 (+ 10929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.184889 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.185005 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:51.189434 (+ 4429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:51.189440 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:51.189444 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:51.189490 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:51.193354 (+ 3864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:51.194229 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:51.194235 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:51.194238 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:51.194420 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:51.194962 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:51.194968 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:51.194973 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:51.195422 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:51.197120 (+ 1698us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:51.197305 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 13:06:51.197644 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260808 13:06:51.206440 23351 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:06:51.206691 23351 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:06:51.198853 22997 master_runner.cc:186] Error getting master registration for 127.1.124.189:34705: OK, instance_id { permanent_uuid: "74ec708a30da43c8b3461289d17f6593" instance_seqno: 1786194410482420 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 13:06:51.214666 22997 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:06:51.214614 23204 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } attempt: 5
Aug 08 13:06:51 dist-test-slave-t3q3 krb5kdc[22979](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194410, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.189@KRBTEST.COM for kudu/127.1.124.188@KRBTEST.COM
I20260808 13:06:51.220413 23217 master_runner.cc:186] Error getting master registration for 127.1.124.189:34705: OK, instance_id { permanent_uuid: "74ec708a30da43c8b3461289d17f6593" instance_seqno: 1786194410482420 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 13:06:51.235153 23217 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:06:51.235822 23322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:51.215913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59250 (local address 127.1.124.188:34251)
0808 13:06:51.216118 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:51.216123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:51.216140 (+ 17us) server_negotiation.cc:407] Connection header received
0808 13:06:51.216207 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:51.216211 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:51.216288 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:51.216399 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:51.217657 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.218761 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.220909 (+ 2148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.222039 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.222214 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:51.227400 (+ 5186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:51.227410 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:51.227417 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:51.227470 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:51.231785 (+ 4315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:51.232809 (+ 1024us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:51.232815 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:51.232819 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:51.232986 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:51.233470 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:51.233475 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:51.233478 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:51.233899 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:51.234076 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:51.235192 (+ 1116us) server_negotiation.cc:299] Negotiation successful
0808 13:06:51.235462 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 13:06:51.236384 23211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:51.215233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:34251 (local address 127.0.0.1:59250)
0808 13:06:51.215583 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:51.215605 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:51.215715 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:51.216641 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:51.216649 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:51.217233 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:06:51.217480 (+ 247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:51.217495 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.218930 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.218935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:51.220675 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:51.220696 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:51.222487 (+ 1791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:51.222496 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:51.222806 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:51.223694 (+ 888us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:06:51.223730 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:06:51.227092 (+ 3362us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:06:51.232015 (+ 4923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:51.232021 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:51.232026 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:51.232643 (+ 617us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:51.233136 (+ 493us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:51.233142 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:51.233145 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:51.233326 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:51.234144 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:06:51.234150 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:06:51.234332 (+ 182us) client_negotiation.cc:769] Sending connection context
0808 13:06:51.234490 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 13:06:51.234813 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220}
W20260808 13:06:51.238694 23372 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:06:51.238847 23372 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:06:51.243999 23204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593: Bootstrap starting.
I20260808 13:06:51.248571 23204 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593: Neither blocks nor log segments found. Creating new log.
I20260808 13:06:51.250077 23204 log.cc:824] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593: Log is configured to *not* fsync() on all Append() calls
I20260808 13:06:51.259461 23204 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593: No bootstrap required, opened a new log
I20260808 13:06:51.271473 23204 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bef1e6073e04d5488c80f87e11020c7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 } } peers { permanent_uuid: "74ec708a30da43c8b3461289d17f6593" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } } peers { permanent_uuid: "60f40066f7da4425a45c721fa1c4cf89" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } }
I20260808 13:06:51.272182 23204 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:06:51.272328 23204 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74ec708a30da43c8b3461289d17f6593, State: Initialized, Role: FOLLOWER
I20260808 13:06:51.273114 23204 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bef1e6073e04d5488c80f87e11020c7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 } } peers { permanent_uuid: "74ec708a30da43c8b3461289d17f6593" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } } peers { permanent_uuid: "60f40066f7da4425a45c721fa1c4cf89" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } }
I20260808 13:06:51.274269 23376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bef1e6073e04d5488c80f87e11020c7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 } } peers { permanent_uuid: "74ec708a30da43c8b3461289d17f6593" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } } peers { permanent_uuid: "60f40066f7da4425a45c721fa1c4cf89" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } } }
I20260808 13:06:51.274802 23376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:06:51.275790 23204 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:06:51.294706 23349 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:06:51.295059 23349 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bef1e6073e04d5488c80f87e11020c7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 } } peers { permanent_uuid: "74ec708a30da43c8b3461289d17f6593" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } } peers { permanent_uuid: "60f40066f7da4425a45c721fa1c4cf89" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } }
I20260808 13:06:51.296896 23349 leader_election.cc:288] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2bef1e6073e04d5488c80f87e11020c7 (127.1.124.190:40701), 74ec708a30da43c8b3461289d17f6593 (127.1.124.189:34705)
W20260808 13:06:51.298245 23243 tablet.cc:2369] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:06:51.297992 23062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60f40066f7da4425a45c721fa1c4cf89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bef1e6073e04d5488c80f87e11020c7" is_pre_election: true
I20260808 13:06:51.298681 23062 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60f40066f7da4425a45c721fa1c4cf89 in term 0.
I20260808 13:06:51.299014 23171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60f40066f7da4425a45c721fa1c4cf89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74ec708a30da43c8b3461289d17f6593" is_pre_election: true
I20260808 13:06:51.299362 23240 leader_election.cc:302] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bef1e6073e04d5488c80f87e11020c7, 60f40066f7da4425a45c721fa1c4cf89; no voters:
I20260808 13:06:51.299578 23171 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 60f40066f7da4425a45c721fa1c4cf89 in term 0.
I20260808 13:06:51.299804 23349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:06:51.299965 23349 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:06:51.300102 23349 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:06:51.303834 23349 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bef1e6073e04d5488c80f87e11020c7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 } } peers { permanent_uuid: "74ec708a30da43c8b3461289d17f6593" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } } peers { permanent_uuid: "60f40066f7da4425a45c721fa1c4cf89" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } }
I20260808 13:06:51.305159 23062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60f40066f7da4425a45c721fa1c4cf89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2bef1e6073e04d5488c80f87e11020c7"
I20260808 13:06:51.305420 23062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:06:51.307283 23110 master_runner.cc:232] Existing masters have differing terms: 0 vs 1
I20260808 13:06:51.307515 23110 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:06:51.309090 23062 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60f40066f7da4425a45c721fa1c4cf89 in term 1.
I20260808 13:06:51.309721 23240 leader_election.cc:302] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bef1e6073e04d5488c80f87e11020c7, 60f40066f7da4425a45c721fa1c4cf89; no voters:
I20260808 13:06:51.312743 23349 leader_election.cc:288] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [CANDIDATE]: Term 1 election: Requested vote from peers 2bef1e6073e04d5488c80f87e11020c7 (127.1.124.190:40701), 74ec708a30da43c8b3461289d17f6593 (127.1.124.189:34705)
I20260808 13:06:51.313329 23349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:06:51.313308 23171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "60f40066f7da4425a45c721fa1c4cf89" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "74ec708a30da43c8b3461289d17f6593"
I20260808 13:06:51.313551 23171 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:06:51.313748 23349 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [term 1 LEADER]: Becoming Leader. State: Replica: 60f40066f7da4425a45c721fa1c4cf89, State: Running, Role: LEADER
I20260808 13:06:51.314355 23349 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bef1e6073e04d5488c80f87e11020c7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 } } peers { permanent_uuid: "74ec708a30da43c8b3461289d17f6593" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } } peers { permanent_uuid: "60f40066f7da4425a45c721fa1c4cf89" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } }
I20260808 13:06:51.317006 23171 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 60f40066f7da4425a45c721fa1c4cf89 in term 1.
I20260808 13:06:51.322257 23392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60f40066f7da4425a45c721fa1c4cf89. Latest consensus state: current_term: 1 leader_uuid: "60f40066f7da4425a45c721fa1c4cf89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bef1e6073e04d5488c80f87e11020c7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 } } peers { permanent_uuid: "74ec708a30da43c8b3461289d17f6593" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } } peers { permanent_uuid: "60f40066f7da4425a45c721fa1c4cf89" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } } }
I20260808 13:06:51.322717 23392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [sys.catalog]: This master's current role is: LEADER
W20260808 13:06:51.323071 23389 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:06:51.323271 23389 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:06:51.323647 23396 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:06:51.325970 23396 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:06:51.349769 23171 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593 [term 1 FOLLOWER]: Refusing update from remote peer 60f40066f7da4425a45c721fa1c4cf89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:06:51.350888 23392 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "74ec708a30da43c8b3461289d17f6593" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:06:51.357599 23062 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7 [term 1 FOLLOWER]: Refusing update from remote peer 60f40066f7da4425a45c721fa1c4cf89: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:06:51.358618 23392 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2bef1e6073e04d5488c80f87e11020c7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:06:51.383297 23376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60f40066f7da4425a45c721fa1c4cf89. Latest consensus state: current_term: 1 leader_uuid: "60f40066f7da4425a45c721fa1c4cf89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bef1e6073e04d5488c80f87e11020c7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 } } peers { permanent_uuid: "74ec708a30da43c8b3461289d17f6593" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } } peers { permanent_uuid: "60f40066f7da4425a45c721fa1c4cf89" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } } }
I20260808 13:06:51.383618 23376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:06:51.392572 23324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60f40066f7da4425a45c721fa1c4cf89. Latest consensus state: current_term: 1 leader_uuid: "60f40066f7da4425a45c721fa1c4cf89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bef1e6073e04d5488c80f87e11020c7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 } } peers { permanent_uuid: "74ec708a30da43c8b3461289d17f6593" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } } peers { permanent_uuid: "60f40066f7da4425a45c721fa1c4cf89" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } } }
I20260808 13:06:51.392894 23324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:06:51.400399 23349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "60f40066f7da4425a45c721fa1c4cf89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bef1e6073e04d5488c80f87e11020c7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 } } peers { permanent_uuid: "74ec708a30da43c8b3461289d17f6593" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } } peers { permanent_uuid: "60f40066f7da4425a45c721fa1c4cf89" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } } }
I20260808 13:06:51.400772 23349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [sys.catalog]: This master's current role is: LEADER
I20260808 13:06:51.403154 23396 catalog_manager.cc:1347] Generated new cluster ID: 8b6d1a4758174a1a87d42217863d0d0a
I20260808 13:06:51.403278 23396 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:06:51.404616 23376 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "60f40066f7da4425a45c721fa1c4cf89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bef1e6073e04d5488c80f87e11020c7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 } } peers { permanent_uuid: "74ec708a30da43c8b3461289d17f6593" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } } peers { permanent_uuid: "60f40066f7da4425a45c721fa1c4cf89" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } } }
I20260808 13:06:51.404866 23376 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:06:51.412377 23392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "60f40066f7da4425a45c721fa1c4cf89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bef1e6073e04d5488c80f87e11020c7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 } } peers { permanent_uuid: "74ec708a30da43c8b3461289d17f6593" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } } peers { permanent_uuid: "60f40066f7da4425a45c721fa1c4cf89" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } } }
I20260808 13:06:51.413607 23392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89 [sys.catalog]: This master's current role is: LEADER
I20260808 13:06:51.420645 23324 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "60f40066f7da4425a45c721fa1c4cf89" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2bef1e6073e04d5488c80f87e11020c7" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 40701 } } peers { permanent_uuid: "74ec708a30da43c8b3461289d17f6593" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 34705 } } peers { permanent_uuid: "60f40066f7da4425a45c721fa1c4cf89" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 34251 } } }
I20260808 13:06:51.420948 23324 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:06:51.440964 23396 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:06:51.443109 23396 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:06:51.464267 23396 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60f40066f7da4425a45c721fa1c4cf89: Generated new TSK 0
I20260808 13:06:51.465562 23396 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:06:52.208946 23351 catalog_manager.cc:1259] Loaded cluster ID: 8b6d1a4758174a1a87d42217863d0d0a
I20260808 13:06:52.209064 23351 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7: loading cluster ID for follower catalog manager: success
I20260808 13:06:52.211535 23351 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7: acquiring CA information for follower catalog manager: success
I20260808 13:06:52.213400 23351 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2bef1e6073e04d5488c80f87e11020c7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 13:06:52.325380 23389 catalog_manager.cc:1259] Loaded cluster ID: 8b6d1a4758174a1a87d42217863d0d0a
I20260808 13:06:52.325489 23389 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593: loading cluster ID for follower catalog manager: success
I20260808 13:06:52.327975 23389 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593: acquiring CA information for follower catalog manager: success
I20260808 13:06:52.329730 23389 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 74ec708a30da43c8b3461289d17f6593: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 13:06:52.634040 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 22997
I20260808 13:06:52.636148 23095 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:06:52.749178 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 22997
I20260808 13:06:52.778899 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 23110
I20260808 13:06:52.780228 23198 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:06:52.887857 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 23110
I20260808 13:06:52.914954 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 23217
I20260808 13:06:52.916208 23306 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:06:53.026137 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 23217
2026-08-08T13:06:53Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (5205 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:06:53 dist-test-slave-t3q3 krb5kdc[23424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:06:53 dist-test-slave-t3q3 krb5kdc[23424](info): set up 2 sockets
Aug 08 13:06:53 dist-test-slave-t3q3 krb5kdc[23424](info): commencing operation
krb5kdc: starting...
W20260808 13:06:54.398435 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.321s user 0.002s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:06:54 dist-test-slave-t3q3 krb5kdc[23424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:06:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:06:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:06:54.701455 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:43541
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:36055
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:43541
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:06:55.052882 23442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:06:55.053288 23442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:06:55.053457 23442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:06:55.060750 23442 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:06:55.060897 23442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:06:55.061013 23442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:06:55.061122 23442 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:06:55.061211 23442 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:06:55.070788 23442 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:36055
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:43541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:43541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:06:55.072183 23442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:06:55.073537 23442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:06:55.085814 23459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:55.085821 23453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:55.085973 23450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:06:55.086823 23442 server_base.cc:1034] running on GCE node
I20260808 13:06:55.087668 23442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:06:55.089124 23442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:06:55.090341 23442 hybrid_clock.cc:648] HybridClock initialized: now 1786194415090285 us; error 58 us; skew 500 ppm
I20260808 13:06:55.090699 23442 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:06:55 dist-test-slave-t3q3 krb5kdc[23424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194415, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:06:55.094095 23442 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:06:55.095646 23442 webserver.cc:460] Webserver started at http://127.1.124.190:41501/ using document root <none> and password file <none>
I20260808 13:06:55.096412 23442 fs_manager.cc:359] Metadata directory not provided
I20260808 13:06:55.096550 23442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:06:55.096892 23442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:06:55.099858 23442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "264c0fda8f7346c7a03c3494c3633ac4"
format_stamp: "Formatted at 2026-08-08 13:06:55 on dist-test-slave-t3q3"
server_key: "5e70a3b581546e2fdc17105702d04164"
server_key_iv: "36da1c7ae7c65e7d647e20667e632539"
server_key_version: "encryptionkey@0"
I20260808 13:06:55.100706 23442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "264c0fda8f7346c7a03c3494c3633ac4"
format_stamp: "Formatted at 2026-08-08 13:06:55 on dist-test-slave-t3q3"
server_key: "5e70a3b581546e2fdc17105702d04164"
server_key_iv: "36da1c7ae7c65e7d647e20667e632539"
server_key_version: "encryptionkey@0"
I20260808 13:06:55.105906 23442 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:06:55.109951 23470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:55.111359 23442 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:06:55.111577 23442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "264c0fda8f7346c7a03c3494c3633ac4"
format_stamp: "Formatted at 2026-08-08 13:06:55 on dist-test-slave-t3q3"
server_key: "5e70a3b581546e2fdc17105702d04164"
server_key_iv: "36da1c7ae7c65e7d647e20667e632539"
server_key_version: "encryptionkey@0"
I20260808 13:06:55.111788 23442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:06:55.125344 23442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:06:55.129550 23442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:06:55.129877 23442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:06:55.149505 23442 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:43541
I20260808 13:06:55.149541 23543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:43541 every 8 connection(s)
I20260808 13:06:55.150959 23442 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:06:55.154796 23544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:06:55.161089 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 23442
I20260808 13:06:55.161355 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:06:55.162264 1522 external_mini_cluster.cc:1428] Setting key 745a899fab7e4405f63d3a7d28fa6b4e
I20260808 13:06:55.166350 23544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4: Bootstrap starting.
I20260808 13:06:55.169540 23544 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4: Neither blocks nor log segments found. Creating new log.
I20260808 13:06:55.170629 23544 log.cc:824] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4: Log is configured to *not* fsync() on all Append() calls
I20260808 13:06:55.173524 23544 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4: No bootstrap required, opened a new log
Aug 08 13:06:55 dist-test-slave-t3q3 krb5kdc[23424](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194414, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:06:55.181918 23544 raft_consensus.cc:348] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264c0fda8f7346c7a03c3494c3633ac4" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43541 } }
I20260808 13:06:55.182340 23544 raft_consensus.cc:374] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:06:55.182477 23544 raft_consensus.cc:729] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 264c0fda8f7346c7a03c3494c3633ac4, State: Initialized, Role: FOLLOWER
I20260808 13:06:55.183143 23544 consensus_queue.cc:260] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264c0fda8f7346c7a03c3494c3633ac4" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43541 } }
I20260808 13:06:55.183421 23544 raft_consensus.cc:388] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:06:55.183579 23544 raft_consensus.cc:482] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:06:55.183781 23544 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:06:55.186134 23547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:55.164460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37644 (local address 127.1.124.190:43541)
0808 13:06:55.165239 (+ 779us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:55.165251 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:55.165292 (+ 41us) server_negotiation.cc:407] Connection header received
0808 13:06:55.167141 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:55.167163 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:55.167522 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:55.167937 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:55.169479 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:55.171019 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:55.172419 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:55.173112 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:55.173309 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:55.178066 (+ 4757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:55.178075 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:55.178091 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:55.178137 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:55.181464 (+ 3327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:55.182277 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:55.182307 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:55.182315 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:55.182552 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:55.182988 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:55.182993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:55.182997 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:55.183750 (+ 753us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:55.184093 (+ 343us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:55.184600 (+ 507us) server_negotiation.cc:299] Negotiation successful
0808 13:06:55.185087 (+ 487us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":571,"thread_start_us":354,"threads_started":1}
I20260808 13:06:55.186527 23544 raft_consensus.cc:504] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264c0fda8f7346c7a03c3494c3633ac4" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43541 } }
I20260808 13:06:55.187211 23544 leader_election.cc:302] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 264c0fda8f7346c7a03c3494c3633ac4; no voters:
I20260808 13:06:55.187943 23544 leader_election.cc:288] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:06:55.188163 23549 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:06:55.189845 23549 raft_consensus.cc:686] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4 [term 1 LEADER]: Becoming Leader. State: Replica: 264c0fda8f7346c7a03c3494c3633ac4, State: Running, Role: LEADER
I20260808 13:06:55.190356 23544 sys_catalog.cc:564] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:06:55.190507 23549 consensus_queue.cc:237] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264c0fda8f7346c7a03c3494c3633ac4" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43541 } }
I20260808 13:06:55.191088 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:06:55.193764 23551 sys_catalog.cc:455] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 264c0fda8f7346c7a03c3494c3633ac4. Latest consensus state: current_term: 1 leader_uuid: "264c0fda8f7346c7a03c3494c3633ac4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264c0fda8f7346c7a03c3494c3633ac4" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43541 } } }
I20260808 13:06:55.194209 23551 sys_catalog.cc:458] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4 [sys.catalog]: This master's current role is: LEADER
I20260808 13:06:55.197158 23550 sys_catalog.cc:455] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "264c0fda8f7346c7a03c3494c3633ac4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "264c0fda8f7346c7a03c3494c3633ac4" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43541 } } }
I20260808 13:06:55.197664 23550 sys_catalog.cc:458] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4 [sys.catalog]: This master's current role is: LEADER
I20260808 13:06:55.200790 23556 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:06:55.206570 23556 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:06:55.216463 23556 catalog_manager.cc:1347] Generated new cluster ID: b46de6cd16704c76a3d9a85acb7d2bd4
I20260808 13:06:55.216658 23556 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:06:55.230230 23556 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:06:55.232194 23556 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:06:55.240499 23556 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 264c0fda8f7346c7a03c3494c3633ac4: Generated new TSK 0
I20260808 13:06:55.241364 23556 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:06:55.254427 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 23442
I20260808 13:06:55.255651 23535 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:06:55.347862 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 23442
2026-08-08T13:06:55Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2315 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:06:55 dist-test-slave-t3q3 krb5kdc[23582](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:06:55 dist-test-slave-t3q3 krb5kdc[23582](info): set up 2 sockets
Aug 08 13:06:55 dist-test-slave-t3q3 krb5kdc[23582](info): commencing operation
krb5kdc: starting...
W20260808 13:06:56.678563 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.285s user 0.004s sys 0.031s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:06:56 dist-test-slave-t3q3 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:06:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:06:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:06:56.979336 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:43305
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:35975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:43305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:06:57.355752 23605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:06:57.356110 23605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:06:57.356289 23605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:06:57.363729 23605 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:06:57.363837 23605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:06:57.363927 23605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:06:57.364042 23605 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:06:57.364180 23605 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:06:57.374625 23605 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:35975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:43305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:43305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:06:57.375963 23605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:06:57.377333 23605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:06:57.389693 23611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:57.389638 23613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:57.389694 23610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:06:57.390661 23605 server_base.cc:1034] running on GCE node
I20260808 13:06:57.391588 23605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:06:57.393255 23605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:06:57.394521 23605 hybrid_clock.cc:648] HybridClock initialized: now 1786194417394469 us; error 58 us; skew 500 ppm
I20260808 13:06:57.394901 23605 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:06:57 dist-test-slave-t3q3 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194417, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:06:57.398352 23605 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:06:57.399969 23605 webserver.cc:460] Webserver started at http://127.1.124.190:45395/ using document root <none> and password file <none>
I20260808 13:06:57.400714 23605 fs_manager.cc:359] Metadata directory not provided
I20260808 13:06:57.400880 23605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:06:57.401204 23605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:06:57.404392 23605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "791268c5f57843bba2907ebd3433e77a"
format_stamp: "Formatted at 2026-08-08 13:06:57 on dist-test-slave-t3q3"
server_key: "ee19eb0b9185b9b68ded49d0f001b21e"
server_key_iv: "6b3f6dc453ccf0e91da6e5259702a0ff"
server_key_version: "encryptionkey@0"
I20260808 13:06:57.405194 23605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "791268c5f57843bba2907ebd3433e77a"
format_stamp: "Formatted at 2026-08-08 13:06:57 on dist-test-slave-t3q3"
server_key: "ee19eb0b9185b9b68ded49d0f001b21e"
server_key_iv: "6b3f6dc453ccf0e91da6e5259702a0ff"
server_key_version: "encryptionkey@0"
I20260808 13:06:57.410590 23605 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:06:57.414201 23624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:57.415520 23605 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:06:57.415717 23605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "791268c5f57843bba2907ebd3433e77a"
format_stamp: "Formatted at 2026-08-08 13:06:57 on dist-test-slave-t3q3"
server_key: "ee19eb0b9185b9b68ded49d0f001b21e"
server_key_iv: "6b3f6dc453ccf0e91da6e5259702a0ff"
server_key_version: "encryptionkey@0"
I20260808 13:06:57.415927 23605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:06:57.429011 23605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:06:57.433588 23605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:06:57.433892 23605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:06:57.453569 23605 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:43305
I20260808 13:06:57.453631 23694 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:43305 every 8 connection(s)
I20260808 13:06:57.455080 23605 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:06:57.458961 23696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:06:57.460561 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 23605
I20260808 13:06:57.460819 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:06:57.461737 1522 external_mini_cluster.cc:1428] Setting key c433c121bbaf939ca7c763fada2b9834
I20260808 13:06:57.474225 23696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a: Bootstrap starting.
Aug 08 13:06:57 dist-test-slave-t3q3 krb5kdc[23582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:06:57.478276 23696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a: Neither blocks nor log segments found. Creating new log.
I20260808 13:06:57.479759 23696 log.cc:824] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a: Log is configured to *not* fsync() on all Append() calls
I20260808 13:06:57.482930 23696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a: No bootstrap required, opened a new log
I20260808 13:06:57.487381 23701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:57.463933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54936 (local address 127.1.124.190:43305)
0808 13:06:57.464853 (+ 920us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:57.464866 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:57.464910 (+ 44us) server_negotiation.cc:407] Connection header received
0808 13:06:57.466488 (+ 1578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:57.466510 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:57.466918 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:57.467265 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:57.469151 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:57.470517 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:57.472470 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:57.473221 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:57.473356 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:57.478744 (+ 5388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:57.478763 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:57.478785 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:57.478852 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:57.482508 (+ 3656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:57.483415 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:57.483422 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:57.483429 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:57.483628 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:57.484294 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:57.484299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:57.484301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:57.484905 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:57.485281 (+ 376us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:57.485801 (+ 520us) server_negotiation.cc:299] Negotiation successful
0808 13:06:57.486256 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":683,"thread_start_us":410,"threads_started":1}
I20260808 13:06:57.492056 23696 raft_consensus.cc:348] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791268c5f57843bba2907ebd3433e77a" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43305 } }
I20260808 13:06:57.492651 23696 raft_consensus.cc:374] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:06:57.492789 23696 raft_consensus.cc:729] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 791268c5f57843bba2907ebd3433e77a, State: Initialized, Role: FOLLOWER
I20260808 13:06:57.493458 23696 consensus_queue.cc:260] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791268c5f57843bba2907ebd3433e77a" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43305 } }
I20260808 13:06:57.493716 23696 raft_consensus.cc:388] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:06:57.493866 23696 raft_consensus.cc:482] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:06:57.494057 23696 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:06:57.496459 23696 raft_consensus.cc:504] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791268c5f57843bba2907ebd3433e77a" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43305 } }
I20260808 13:06:57.497043 23696 leader_election.cc:302] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 791268c5f57843bba2907ebd3433e77a; no voters:
I20260808 13:06:57.497706 23696 leader_election.cc:288] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:06:57.497874 23704 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:06:57.498582 23704 raft_consensus.cc:686] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 1 LEADER]: Becoming Leader. State: Replica: 791268c5f57843bba2907ebd3433e77a, State: Running, Role: LEADER
I20260808 13:06:57.499315 23704 consensus_queue.cc:237] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791268c5f57843bba2907ebd3433e77a" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43305 } }
I20260808 13:06:57.499970 23696 sys_catalog.cc:564] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:06:57.502578 23705 sys_catalog.cc:455] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "791268c5f57843bba2907ebd3433e77a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791268c5f57843bba2907ebd3433e77a" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43305 } } }
I20260808 13:06:57.502568 23706 sys_catalog.cc:455] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 791268c5f57843bba2907ebd3433e77a. Latest consensus state: current_term: 1 leader_uuid: "791268c5f57843bba2907ebd3433e77a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791268c5f57843bba2907ebd3433e77a" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43305 } } }
I20260808 13:06:57.503046 23705 sys_catalog.cc:458] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [sys.catalog]: This master's current role is: LEADER
I20260808 13:06:57.503046 23706 sys_catalog.cc:458] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [sys.catalog]: This master's current role is: LEADER
I20260808 13:06:57.505234 23709 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:06:57.511588 23709 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:06:57.521334 23709 catalog_manager.cc:1347] Generated new cluster ID: 758e54337868452d9506f08c2a124c67
I20260808 13:06:57.521464 23709 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:06:57.533536 23709 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:06:57.534929 23709 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:06:57.546867 23709 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a: Generated new TSK 0
I20260808 13:06:57.547765 23709 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.124.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
I20260808 13:06:57.620613 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:43305
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:35975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:06:57.973744 23735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:06:57.974026 23735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:06:57.974189 23735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:06:57.981688 23735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:06:57.981794 23735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:06:57.981968 23735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:06:57.991791 23735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:35975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:43305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:06:57.993168 23735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:06:57.994498 23735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:06:58.008309 23741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:58.008271 23744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:58.008286 23742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:06:58.009341 23735 server_base.cc:1034] running on GCE node
I20260808 13:06:58.009904 23735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:06:58.010807 23735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:06:58.012022 23735 hybrid_clock.cc:648] HybridClock initialized: now 1786194418011964 us; error 53 us; skew 500 ppm
I20260808 13:06:58.012382 23735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:06:58 dist-test-slave-t3q3 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194418, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:06:58.015457 23735 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:06:58.016937 23735 webserver.cc:460] Webserver started at http://127.1.124.129:38565/ using document root <none> and password file <none>
I20260808 13:06:58.017609 23735 fs_manager.cc:359] Metadata directory not provided
I20260808 13:06:58.017714 23735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:06:58.017967 23735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:06:58.021155 23735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "4a485f59cf414ba0bb3fedb581b344dd"
format_stamp: "Formatted at 2026-08-08 13:06:58 on dist-test-slave-t3q3"
server_key: "45a6a534451f30fb91ee2eec3f4ba46f"
server_key_iv: "3f4e95820fcfb25c5dcfad2283cccb71"
server_key_version: "encryptionkey@0"
I20260808 13:06:58.021955 23735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "4a485f59cf414ba0bb3fedb581b344dd"
format_stamp: "Formatted at 2026-08-08 13:06:58 on dist-test-slave-t3q3"
server_key: "45a6a534451f30fb91ee2eec3f4ba46f"
server_key_iv: "3f4e95820fcfb25c5dcfad2283cccb71"
server_key_version: "encryptionkey@0"
I20260808 13:06:58.027513 23735 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260808 13:06:58.031358 23753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:58.032866 23735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:06:58.033106 23735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "4a485f59cf414ba0bb3fedb581b344dd"
format_stamp: "Formatted at 2026-08-08 13:06:58 on dist-test-slave-t3q3"
server_key: "45a6a534451f30fb91ee2eec3f4ba46f"
server_key_iv: "3f4e95820fcfb25c5dcfad2283cccb71"
server_key_version: "encryptionkey@0"
I20260808 13:06:58.033329 23735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:06:58.052449 23735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:06:58.057240 23735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:06:58.057571 23735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:06:58.058607 23735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:06:58.060137 23735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:06:58.060240 23735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:58.060395 23735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:06:58.060469 23735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:58.095170 23735 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:35215
I20260808 13:06:58.095269 23911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:35215 every 8 connection(s)
I20260808 13:06:58.096653 23735 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:06:58.102932 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 23735
I20260808 13:06:58.103185 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:06:58.104004 1522 external_mini_cluster.cc:1428] Setting key 6f8c8f1e6f351ad1bbc404c615618e45
Aug 08 13:06:58 dist-test-slave-t3q3 krb5kdc[23582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194418, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:06:58.121241 23914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:58.099701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:51327 (local address 127.1.124.190:43305)
0808 13:06:58.100261 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:58.100267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:58.101424 (+ 1157us) server_negotiation.cc:407] Connection header received
0808 13:06:58.103299 (+ 1875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:58.103305 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:58.103400 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:58.103530 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:58.105285 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:58.106400 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:58.109111 (+ 2711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:58.109501 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:58.109604 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:58.114116 (+ 4512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:58.114126 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:58.114133 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:58.114180 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:58.117912 (+ 3732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:58.118750 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:58.118754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:58.118756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:58.118886 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:58.119400 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:58.119404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:58.119406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:58.119693 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:58.119805 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:58.120792 (+ 987us) server_negotiation.cc:299] Negotiation successful
0808 13:06:58.120963 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":233,"threads_started":1}
I20260808 13:06:58.122828 23915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:58.100362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.129:51327)
0808 13:06:58.101224 (+ 862us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:58.101276 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:58.102961 (+ 1685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:58.103752 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:58.103760 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:58.104394 (+ 634us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:06:58.105094 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:58.105121 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:58.106586 (+ 1465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:58.106592 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:58.108941 (+ 2349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:58.108961 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:58.109756 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:58.109766 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:58.110098 (+ 332us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:58.110868 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:06:58.110910 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:06:58.113857 (+ 2947us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:06:58.118067 (+ 4210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:58.118074 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:58.118089 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:58.118608 (+ 519us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:58.118998 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:58.119004 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:58.119010 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:58.119281 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:58.119831 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:06:58.119842 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:06:58.120369 (+ 527us) client_negotiation.cc:769] Sending connection context
0808 13:06:58.120693 (+ 324us) client_negotiation.cc:240] Negotiation successful
0808 13:06:58.121302 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":296,"threads_started":1}
I20260808 13:06:58.124622 23912 heartbeater.cc:347] Connected to a master server at 127.1.124.190:43305
I20260808 13:06:58.124948 23912 heartbeater.cc:464] Registering TS with master...
I20260808 13:06:58.125706 23912 heartbeater.cc:511] Master 127.1.124.190:43305 requested a full tablet report, sending...
I20260808 13:06:58.127816 23649 ts_manager.cc:194] Registered new tserver with Master: 4a485f59cf414ba0bb3fedb581b344dd (127.1.124.129:35215)
I20260808 13:06:58.129311 23649 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:51327
WARNING: no policy specified for kudu/127.1.124.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
I20260808 13:06:58.172904 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:43305
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:35975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:06:58.518718 23919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:06:58.519039 23919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:06:58.519251 23919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:06:58.526541 23919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:06:58.526684 23919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:06:58.526871 23919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:06:58.536362 23919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:35975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:43305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:06:58.537731 23919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:06:58.539140 23919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:06:58.552570 23928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:58.552613 23930 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:58.552651 23934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:06:58.553510 23919 server_base.cc:1034] running on GCE node
I20260808 13:06:58.554132 23919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:06:58.555167 23919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:06:58.556393 23919 hybrid_clock.cc:648] HybridClock initialized: now 1786194418556356 us; error 58 us; skew 500 ppm
I20260808 13:06:58.556725 23919 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:06:58 dist-test-slave-t3q3 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194418, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:06:58.562686 23919 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:06:58.564325 23919 webserver.cc:460] Webserver started at http://127.1.124.130:36307/ using document root <none> and password file <none>
I20260808 13:06:58.565032 23919 fs_manager.cc:359] Metadata directory not provided
I20260808 13:06:58.565212 23919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:06:58.565526 23919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:06:58.568621 23919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "6acc4ef52963473d993cc66899bbe5cb"
format_stamp: "Formatted at 2026-08-08 13:06:58 on dist-test-slave-t3q3"
server_key: "ca6c479dc37a2c95cc652c35527a960d"
server_key_iv: "3f2df92286f2ebba14fb05ff034a34c3"
server_key_version: "encryptionkey@0"
I20260808 13:06:58.569428 23919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "6acc4ef52963473d993cc66899bbe5cb"
format_stamp: "Formatted at 2026-08-08 13:06:58 on dist-test-slave-t3q3"
server_key: "ca6c479dc37a2c95cc652c35527a960d"
server_key_iv: "3f2df92286f2ebba14fb05ff034a34c3"
server_key_version: "encryptionkey@0"
I20260808 13:06:58.574738 23919 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:06:58.578224 23942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:58.579675 23919 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:06:58.579877 23919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "6acc4ef52963473d993cc66899bbe5cb"
format_stamp: "Formatted at 2026-08-08 13:06:58 on dist-test-slave-t3q3"
server_key: "ca6c479dc37a2c95cc652c35527a960d"
server_key_iv: "3f2df92286f2ebba14fb05ff034a34c3"
server_key_version: "encryptionkey@0"
I20260808 13:06:58.580163 23919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:06:58.602530 23919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:06:58.606761 23919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:06:58.607064 23919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:06:58.608039 23919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:06:58.609380 23919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:06:58.609491 23919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:58.609647 23919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:06:58.609721 23919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:58.644119 23919 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:37813
I20260808 13:06:58.644174 24131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:37813 every 8 connection(s)
I20260808 13:06:58.645473 23919 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:06:58.651420 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 23919
I20260808 13:06:58.651641 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:06:58.652336 1522 external_mini_cluster.cc:1428] Setting key e0466db7e95006bfe64f061f7850bc27
Aug 08 13:06:58 dist-test-slave-t3q3 krb5kdc[23582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194418, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:06:58.668421 24135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:58.648043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:34345 (local address 127.1.124.190:43305)
0808 13:06:58.648547 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:58.648552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:58.649522 (+ 970us) server_negotiation.cc:407] Connection header received
0808 13:06:58.651660 (+ 2138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:58.651668 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:58.651792 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:58.651888 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:58.653763 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:58.654809 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:58.657679 (+ 2870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:58.658120 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:58.658221 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:58.662577 (+ 4356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:58.662582 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:58.662585 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:58.662621 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:58.665219 (+ 2598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:58.665975 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:58.665984 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:58.665987 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:58.666118 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:58.666562 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:58.666572 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:58.666574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:58.666847 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:58.667005 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:58.667766 (+ 761us) server_negotiation.cc:299] Negotiation successful
0808 13:06:58.668003 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":246,"threads_started":1}
I20260808 13:06:58.669450 24136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:58.648556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.130:34345)
0808 13:06:58.649337 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:58.649387 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:58.651311 (+ 1924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:58.652204 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:58.652216 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:58.652998 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:06:58.653578 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:58.653601 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:58.654977 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:58.654982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:58.657499 (+ 2517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:58.657519 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:58.658355 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:58.658364 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:58.658706 (+ 342us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:58.659490 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:06:58.659524 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:06:58.662321 (+ 2797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:06:58.665366 (+ 3045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:58.665376 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:58.665394 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:58.665839 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:58.666236 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:58.666241 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:58.666244 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:58.666437 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:58.666981 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:06:58.666989 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:06:58.667342 (+ 353us) client_negotiation.cc:769] Sending connection context
0808 13:06:58.667653 (+ 311us) client_negotiation.cc:240] Negotiation successful
0808 13:06:58.668249 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":319,"threads_started":1}
I20260808 13:06:58.670984 24132 heartbeater.cc:347] Connected to a master server at 127.1.124.190:43305
I20260808 13:06:58.671366 24132 heartbeater.cc:464] Registering TS with master...
I20260808 13:06:58.672286 24132 heartbeater.cc:511] Master 127.1.124.190:43305 requested a full tablet report, sending...
I20260808 13:06:58.674058 23649 ts_manager.cc:194] Registered new tserver with Master: 6acc4ef52963473d993cc66899bbe5cb (127.1.124.130:37813)
I20260808 13:06:58.675364 23649 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:34345
WARNING: no policy specified for kudu/127.1.124.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
I20260808 13:06:58.720510 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:43305
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:35975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:06:59.078204 24140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:06:59.078513 24140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:06:59.078624 24140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:06:59.086711 24140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:06:59.086844 24140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:06:59.086961 24140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:06:59.098568 24140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:35975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:43305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:06:59.100302 24140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:06:59.102133 24140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:06:59.116235 24148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:59.116186 24151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:06:59.116286 24149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:06:59.121475 24140 server_base.cc:1034] running on GCE node
I20260808 13:06:59.122124 24140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:06:59.123086 24140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:06:59.124361 24140 hybrid_clock.cc:648] HybridClock initialized: now 1786194419124322 us; error 70 us; skew 500 ppm
I20260808 13:06:59.124707 24140 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:06:59 dist-test-slave-t3q3 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194419, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:06:59.128021 24140 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:06:59.129760 24140 webserver.cc:460] Webserver started at http://127.1.124.131:42503/ using document root <none> and password file <none>
I20260808 13:06:59.130506 24140 fs_manager.cc:359] Metadata directory not provided
I20260808 13:06:59.130657 24140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:06:59.130975 24140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:06:59.132421 23912 heartbeater.cc:503] Master 127.1.124.190:43305 was elected leader, sending a full tablet report...
I20260808 13:06:59.134513 24140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "df1ac1b971934f54a95716037b089d34"
format_stamp: "Formatted at 2026-08-08 13:06:59 on dist-test-slave-t3q3"
server_key: "28463d66a8ebdeb2df6d4f46434dc890"
server_key_iv: "bedac146c41773fde0c222688632ba23"
server_key_version: "encryptionkey@0"
I20260808 13:06:59.135331 24140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "df1ac1b971934f54a95716037b089d34"
format_stamp: "Formatted at 2026-08-08 13:06:59 on dist-test-slave-t3q3"
server_key: "28463d66a8ebdeb2df6d4f46434dc890"
server_key_iv: "bedac146c41773fde0c222688632ba23"
server_key_version: "encryptionkey@0"
I20260808 13:06:59.141245 24140 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 13:06:59.145069 24160 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:59.146763 24140 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:06:59.147001 24140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "df1ac1b971934f54a95716037b089d34"
format_stamp: "Formatted at 2026-08-08 13:06:59 on dist-test-slave-t3q3"
server_key: "28463d66a8ebdeb2df6d4f46434dc890"
server_key_iv: "bedac146c41773fde0c222688632ba23"
server_key_version: "encryptionkey@0"
I20260808 13:06:59.147264 24140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:06:59.167884 24140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:06:59.173344 24140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:06:59.173707 24140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:06:59.174902 24140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:06:59.176599 24140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:06:59.176717 24140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:59.176890 24140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:06:59.176980 24140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:06:59.217546 24140 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:44529
I20260808 13:06:59.217587 24328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:44529 every 8 connection(s)
I20260808 13:06:59.219074 24140 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:06:59.229133 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 24140
I20260808 13:06:59.229362 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:06:59.229883 1522 external_mini_cluster.cc:1428] Setting key 026c174c82c1f498f547656c6967e2ba
Aug 08 13:06:59 dist-test-slave-t3q3 krb5kdc[23582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194419, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:06:59.242308 24331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:59.222172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:52229 (local address 127.1.124.190:43305)
0808 13:06:59.222702 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:59.222706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:59.223905 (+ 1199us) server_negotiation.cc:407] Connection header received
0808 13:06:59.226094 (+ 2189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:59.226100 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:59.226208 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:59.226344 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:59.228015 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.229016 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.231147 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.231699 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.231862 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:59.236161 (+ 4299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:59.236167 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:59.236170 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:59.236203 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:59.239086 (+ 2883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:59.239874 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:59.239880 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:59.239885 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:59.240018 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:59.240493 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:59.240499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:59.240502 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:59.240788 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:59.240940 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:59.241680 (+ 740us) server_negotiation.cc:299] Negotiation successful
0808 13:06:59.241913 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":261,"threads_started":1}
I20260808 13:06:59.243436 24332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:59.222851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.131:52229)
0808 13:06:59.223702 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:59.223754 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:59.225788 (+ 2034us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:59.226622 (+ 834us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:59.226632 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:59.227212 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:06:59.227822 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:59.227842 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.229205 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.229230 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:59.230975 (+ 1745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:59.230989 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.231996 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.232031 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:59.232411 (+ 380us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:59.233360 (+ 949us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:06:59.233394 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:06:59.235900 (+ 2506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:06:59.239259 (+ 3359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:59.239269 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:59.239288 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:59.239735 (+ 447us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:59.240158 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:06:59.240163 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:06:59.240167 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:06:59.240366 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:06:59.240932 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:06:59.240942 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:06:59.241308 (+ 366us) client_negotiation.cc:769] Sending connection context
0808 13:06:59.241596 (+ 288us) client_negotiation.cc:240] Negotiation successful
0808 13:06:59.242171 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":334,"threads_started":1}
I20260808 13:06:59.245398 24329 heartbeater.cc:347] Connected to a master server at 127.1.124.190:43305
I20260808 13:06:59.245755 24329 heartbeater.cc:464] Registering TS with master...
I20260808 13:06:59.246724 24329 heartbeater.cc:511] Master 127.1.124.190:43305 requested a full tablet report, sending...
I20260808 13:06:59.248332 23646 ts_manager.cc:194] Registered new tserver with Master: df1ac1b971934f54a95716037b089d34 (127.1.124.131:44529)
I20260808 13:06:59.249279 23646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:52229
I20260808 13:06:59.258961 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:06:59.277745 24331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:59.264190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54940 (local address 127.1.124.190:43305)
0808 13:06:59.264445 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:59.264454 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:59.264809 (+ 355us) server_negotiation.cc:407] Connection header received
0808 13:06:59.264978 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:59.264981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:59.265083 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:59.265179 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:59.266861 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.267683 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.269167 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.269618 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.269729 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:59.272394 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:59.272399 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:59.272403 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:59.272436 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:59.275095 (+ 2659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:59.275744 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:59.275749 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:59.275751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:59.275900 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:59.276340 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:59.276344 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:59.276346 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:59.276683 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:59.276866 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:59.277184 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 13:06:59.277363 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 13:06:59.286366 23646 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54940:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:06:59.293334 23646 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:06:59.331346 24345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:59.306611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:37813 (local address 127.0.0.1:34886)
0808 13:06:59.321243 (+ 14632us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:59.321265 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:59.321404 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:59.323642 (+ 2238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:59.323650 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:59.323689 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:06:59.324125 (+ 436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:59.324142 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.326341 (+ 2199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.326347 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:59.328718 (+ 2371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:59.328732 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.330215 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.330222 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:59.330438 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:59.330503 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 13:06:59.330633 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 13:06:59.330849 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14440,"mutex_wait_us":3,"thread_start_us":13400,"threads_started":1}
I20260808 13:06:59.332476 24352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:59.306930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34886 (local address 127.1.124.130:37813)
0808 13:06:59.323034 (+ 16104us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:59.323040 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:59.323065 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:06:59.323210 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:59.323218 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:59.323421 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:59.323651 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:06:59.324364 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.326150 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.328894 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.330001 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.330167 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:59.330391 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:59.331844 (+ 1453us) server_negotiation.cc:299] Negotiation successful
0808 13:06:59.332141 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15952,"thread_start_us":15805,"threads_started":1}
I20260808 13:06:59.332723 24349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:59.306483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:35215 (local address 127.0.0.1:57708)
0808 13:06:59.320564 (+ 14081us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:59.320634 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:59.320857 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:59.324880 (+ 4023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:59.324884 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:59.324911 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:06:59.325474 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:59.325488 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.327601 (+ 2113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.327614 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:59.329905 (+ 2291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:59.329914 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.331901 (+ 1987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.331905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:59.332021 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:59.332043 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 13:06:59.332202 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 13:06:59.332325 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13736,"mutex_wait_us":23,"thread_start_us":15025,"threads_started":1}
I20260808 13:06:59.333700 24354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:59.309512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:44529 (local address 127.0.0.1:57960)
0808 13:06:59.322305 (+ 12793us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:59.322327 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:59.322457 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:59.324855 (+ 2398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:59.324861 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:59.324889 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:06:59.325429 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:59.325444 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.327335 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.327342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:59.329886 (+ 2544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:59.329902 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.332883 (+ 2981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.332891 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:59.333154 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:59.333200 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 13:06:59.333295 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 13:06:59.333386 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12619,"mutex_wait_us":140,"thread_start_us":12456,"threads_started":1}
I20260808 13:06:59.333906 24355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:59.310826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57960 (local address 127.1.124.131:44529)
0808 13:06:59.323827 (+ 13001us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:59.323841 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:59.323887 (+ 46us) server_negotiation.cc:407] Connection header received
0808 13:06:59.324014 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:59.324026 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:59.324304 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:59.324546 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:06:59.325669 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.327196 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.330623 (+ 3427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.332314 (+ 1691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.332459 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:59.332732 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:59.333372 (+ 640us) server_negotiation.cc:299] Negotiation successful
0808 13:06:59.333539 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12773,"thread_start_us":12565,"threads_started":1}
I20260808 13:06:59.332754 24350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:59.306723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57708 (local address 127.1.124.129:35215)
0808 13:06:59.324389 (+ 17666us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:59.324395 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:59.324415 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:06:59.324485 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:59.324490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:59.324642 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:59.324875 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:06:59.325699 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.327430 (+ 1731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.330081 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.331408 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.331576 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:59.331792 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:59.332256 (+ 464us) server_negotiation.cc:299] Negotiation successful
0808 13:06:59.332419 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17493,"thread_start_us":17319,"threads_started":1}
I20260808 13:06:59.336974 24028 tablet_service.cc:1467] Processing CreateTablet for tablet e75e025b0bb9463daf1fbffe11b17095 (DEFAULT_TABLE table=test-table [id=470bb72a9fbf4aa7859e57a7d3a12208]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:06:59.337801 24230 tablet_service.cc:1467] Processing CreateTablet for tablet e75e025b0bb9463daf1fbffe11b17095 (DEFAULT_TABLE table=test-table [id=470bb72a9fbf4aa7859e57a7d3a12208]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:06:59.337802 23814 tablet_service.cc:1467] Processing CreateTablet for tablet e75e025b0bb9463daf1fbffe11b17095 (DEFAULT_TABLE table=test-table [id=470bb72a9fbf4aa7859e57a7d3a12208]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:06:59.338411 24028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e75e025b0bb9463daf1fbffe11b17095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:06:59.339282 24230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e75e025b0bb9463daf1fbffe11b17095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:06:59.339282 23814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e75e025b0bb9463daf1fbffe11b17095. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:06:59.350106 24357 tablet_bootstrap.cc:492] T e75e025b0bb9463daf1fbffe11b17095 P 6acc4ef52963473d993cc66899bbe5cb: Bootstrap starting.
I20260808 13:06:59.353658 24357 tablet_bootstrap.cc:654] T e75e025b0bb9463daf1fbffe11b17095 P 6acc4ef52963473d993cc66899bbe5cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:06:59.354749 24357 log.cc:824] T e75e025b0bb9463daf1fbffe11b17095 P 6acc4ef52963473d993cc66899bbe5cb: Log is configured to *not* fsync() on all Append() calls
I20260808 13:06:59.355397 24358 tablet_bootstrap.cc:492] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd: Bootstrap starting.
I20260808 13:06:59.356398 24359 tablet_bootstrap.cc:492] T e75e025b0bb9463daf1fbffe11b17095 P df1ac1b971934f54a95716037b089d34: Bootstrap starting.
I20260808 13:06:59.359103 24357 tablet_bootstrap.cc:492] T e75e025b0bb9463daf1fbffe11b17095 P 6acc4ef52963473d993cc66899bbe5cb: No bootstrap required, opened a new log
I20260808 13:06:59.359400 24357 ts_tablet_manager.cc:1397] T e75e025b0bb9463daf1fbffe11b17095 P 6acc4ef52963473d993cc66899bbe5cb: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260808 13:06:59.360388 24359 tablet_bootstrap.cc:654] T e75e025b0bb9463daf1fbffe11b17095 P df1ac1b971934f54a95716037b089d34: Neither blocks nor log segments found. Creating new log.
I20260808 13:06:59.360651 24358 tablet_bootstrap.cc:654] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd: Neither blocks nor log segments found. Creating new log.
I20260808 13:06:59.361764 24359 log.cc:824] T e75e025b0bb9463daf1fbffe11b17095 P df1ac1b971934f54a95716037b089d34: Log is configured to *not* fsync() on all Append() calls
I20260808 13:06:59.361764 24358 log.cc:824] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd: Log is configured to *not* fsync() on all Append() calls
I20260808 13:06:59.364992 24359 tablet_bootstrap.cc:492] T e75e025b0bb9463daf1fbffe11b17095 P df1ac1b971934f54a95716037b089d34: No bootstrap required, opened a new log
I20260808 13:06:59.365006 24358 tablet_bootstrap.cc:492] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd: No bootstrap required, opened a new log
I20260808 13:06:59.365264 24358 ts_tablet_manager.cc:1397] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260808 13:06:59.365288 24359 ts_tablet_manager.cc:1397] T e75e025b0bb9463daf1fbffe11b17095 P df1ac1b971934f54a95716037b089d34: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260808 13:06:59.368271 24357 raft_consensus.cc:348] T e75e025b0bb9463daf1fbffe11b17095 P 6acc4ef52963473d993cc66899bbe5cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a485f59cf414ba0bb3fedb581b344dd" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35215 } } peers { permanent_uuid: "6acc4ef52963473d993cc66899bbe5cb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37813 } } peers { permanent_uuid: "df1ac1b971934f54a95716037b089d34" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44529 } }
I20260808 13:06:59.368746 24357 raft_consensus.cc:374] T e75e025b0bb9463daf1fbffe11b17095 P 6acc4ef52963473d993cc66899bbe5cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:06:59.368898 24357 raft_consensus.cc:729] T e75e025b0bb9463daf1fbffe11b17095 P 6acc4ef52963473d993cc66899bbe5cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6acc4ef52963473d993cc66899bbe5cb, State: Initialized, Role: FOLLOWER
I20260808 13:06:59.369568 24357 consensus_queue.cc:260] T e75e025b0bb9463daf1fbffe11b17095 P 6acc4ef52963473d993cc66899bbe5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a485f59cf414ba0bb3fedb581b344dd" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35215 } } peers { permanent_uuid: "6acc4ef52963473d993cc66899bbe5cb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37813 } } peers { permanent_uuid: "df1ac1b971934f54a95716037b089d34" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44529 } }
I20260808 13:06:59.370748 24132 heartbeater.cc:503] Master 127.1.124.190:43305 was elected leader, sending a full tablet report...
I20260808 13:06:59.371333 24357 ts_tablet_manager.cc:1428] T e75e025b0bb9463daf1fbffe11b17095 P 6acc4ef52963473d993cc66899bbe5cb: Time spent starting tablet: real 0.012s user 0.008s sys 0.005s
I20260808 13:06:59.374470 24359 raft_consensus.cc:348] T e75e025b0bb9463daf1fbffe11b17095 P df1ac1b971934f54a95716037b089d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a485f59cf414ba0bb3fedb581b344dd" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35215 } } peers { permanent_uuid: "6acc4ef52963473d993cc66899bbe5cb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37813 } } peers { permanent_uuid: "df1ac1b971934f54a95716037b089d34" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44529 } }
I20260808 13:06:59.374833 24359 raft_consensus.cc:374] T e75e025b0bb9463daf1fbffe11b17095 P df1ac1b971934f54a95716037b089d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:06:59.374954 24359 raft_consensus.cc:729] T e75e025b0bb9463daf1fbffe11b17095 P df1ac1b971934f54a95716037b089d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df1ac1b971934f54a95716037b089d34, State: Initialized, Role: FOLLOWER
I20260808 13:06:59.375609 24359 consensus_queue.cc:260] T e75e025b0bb9463daf1fbffe11b17095 P df1ac1b971934f54a95716037b089d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a485f59cf414ba0bb3fedb581b344dd" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35215 } } peers { permanent_uuid: "6acc4ef52963473d993cc66899bbe5cb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37813 } } peers { permanent_uuid: "df1ac1b971934f54a95716037b089d34" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44529 } }
I20260808 13:06:59.376736 24329 heartbeater.cc:503] Master 127.1.124.190:43305 was elected leader, sending a full tablet report...
I20260808 13:06:59.377367 24359 ts_tablet_manager.cc:1428] T e75e025b0bb9463daf1fbffe11b17095 P df1ac1b971934f54a95716037b089d34: Time spent starting tablet: real 0.012s user 0.006s sys 0.006s
I20260808 13:06:59.377553 24358 raft_consensus.cc:348] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a485f59cf414ba0bb3fedb581b344dd" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35215 } } peers { permanent_uuid: "6acc4ef52963473d993cc66899bbe5cb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37813 } } peers { permanent_uuid: "df1ac1b971934f54a95716037b089d34" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44529 } }
I20260808 13:06:59.377947 24358 raft_consensus.cc:374] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:06:59.378057 24358 raft_consensus.cc:729] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a485f59cf414ba0bb3fedb581b344dd, State: Initialized, Role: FOLLOWER
I20260808 13:06:59.378710 24358 consensus_queue.cc:260] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a485f59cf414ba0bb3fedb581b344dd" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35215 } } peers { permanent_uuid: "6acc4ef52963473d993cc66899bbe5cb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37813 } } peers { permanent_uuid: "df1ac1b971934f54a95716037b089d34" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44529 } }
I20260808 13:06:59.380656 24358 ts_tablet_manager.cc:1428] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
W20260808 13:06:59.397601 24134 tablet.cc:2369] T e75e025b0bb9463daf1fbffe11b17095 P 6acc4ef52963473d993cc66899bbe5cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:06:59.413580 24366 raft_consensus.cc:482] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:06:59.413887 24366 raft_consensus.cc:504] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a485f59cf414ba0bb3fedb581b344dd" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35215 } } peers { permanent_uuid: "6acc4ef52963473d993cc66899bbe5cb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37813 } } peers { permanent_uuid: "df1ac1b971934f54a95716037b089d34" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44529 } }
I20260808 13:06:59.415656 24366 leader_election.cc:288] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6acc4ef52963473d993cc66899bbe5cb (127.1.124.130:37813), df1ac1b971934f54a95716037b089d34 (127.1.124.131:44529)
I20260808 13:06:59.423481 24352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:59.415977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:43663 (local address 127.1.124.130:37813)
0808 13:06:59.416247 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:59.416256 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:59.417584 (+ 1328us) server_negotiation.cc:407] Connection header received
0808 13:06:59.417780 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:59.417787 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:59.417863 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:59.417984 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:06:59.418737 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.419613 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.421728 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.422492 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.422647 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:59.422738 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:59.423069 (+ 331us) server_negotiation.cc:299] Negotiation successful
0808 13:06:59.423209 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105}
I20260808 13:06:59.423485 24355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:59.416200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:57253 (local address 127.1.124.131:44529)
0808 13:06:59.416421 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:59.416429 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:59.417634 (+ 1205us) server_negotiation.cc:407] Connection header received
0808 13:06:59.417842 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:59.417847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:59.417926 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:59.418021 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:06:59.418816 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.419628 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.421760 (+ 2132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.422509 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.422699 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:59.422771 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:59.423123 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 13:06:59.423242 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 13:06:59.423619 24371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:59.416021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:37813 (local address 127.1.124.129:43663)
0808 13:06:59.417425 (+ 1404us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:59.417453 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:59.417620 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:59.418038 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:59.418043 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:59.418068 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:06:59.418551 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:59.418565 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.419743 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.419749 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:59.421560 (+ 1811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:59.421572 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.422723 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.422730 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:59.422904 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:59.422927 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 13:06:59.423013 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 13:06:59.423170 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1271,"mutex_wait_us":94,"thread_start_us":473,"threads_started":1}
I20260808 13:06:59.423676 24370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:59.416033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:44529 (local address 127.1.124.129:57253)
0808 13:06:59.417461 (+ 1428us) negotiation.cc:107] Waiting for socket to connect
0808 13:06:59.417476 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:06:59.417698 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:06:59.418191 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:06:59.418195 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:06:59.418237 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:06:59.418664 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:59.418677 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.419804 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.419807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:59.421587 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:06:59.421599 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.422766 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.422773 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:06:59.422943 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:59.422965 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 13:06:59.423081 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 13:06:59.423221 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1326,"mutex_wait_us":99,"spinlock_wait_cycles":22912,"thread_start_us":646,"threads_started":1}
I20260808 13:06:59.424796 24260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e75e025b0bb9463daf1fbffe11b17095" candidate_uuid: "4a485f59cf414ba0bb3fedb581b344dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df1ac1b971934f54a95716037b089d34" is_pre_election: true
I20260808 13:06:59.424798 24056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e75e025b0bb9463daf1fbffe11b17095" candidate_uuid: "4a485f59cf414ba0bb3fedb581b344dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6acc4ef52963473d993cc66899bbe5cb" is_pre_election: true
I20260808 13:06:59.425392 24056 raft_consensus.cc:2454] T e75e025b0bb9463daf1fbffe11b17095 P 6acc4ef52963473d993cc66899bbe5cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a485f59cf414ba0bb3fedb581b344dd in term 0.
I20260808 13:06:59.425392 24260 raft_consensus.cc:2454] T e75e025b0bb9463daf1fbffe11b17095 P df1ac1b971934f54a95716037b089d34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a485f59cf414ba0bb3fedb581b344dd in term 0.
I20260808 13:06:59.426082 23761 leader_election.cc:302] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a485f59cf414ba0bb3fedb581b344dd, df1ac1b971934f54a95716037b089d34; no voters:
I20260808 13:06:59.426440 24366 raft_consensus.cc:2781] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:06:59.426577 24366 raft_consensus.cc:482] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:06:59.426740 24366 raft_consensus.cc:3037] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:06:59.429103 24366 raft_consensus.cc:504] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a485f59cf414ba0bb3fedb581b344dd" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35215 } } peers { permanent_uuid: "6acc4ef52963473d993cc66899bbe5cb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37813 } } peers { permanent_uuid: "df1ac1b971934f54a95716037b089d34" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44529 } }
I20260808 13:06:59.429699 24366 leader_election.cc:288] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd [CANDIDATE]: Term 1 election: Requested vote from peers 6acc4ef52963473d993cc66899bbe5cb (127.1.124.130:37813), df1ac1b971934f54a95716037b089d34 (127.1.124.131:44529)
I20260808 13:06:59.430179 24056 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e75e025b0bb9463daf1fbffe11b17095" candidate_uuid: "4a485f59cf414ba0bb3fedb581b344dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6acc4ef52963473d993cc66899bbe5cb"
I20260808 13:06:59.430226 24260 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e75e025b0bb9463daf1fbffe11b17095" candidate_uuid: "4a485f59cf414ba0bb3fedb581b344dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df1ac1b971934f54a95716037b089d34"
I20260808 13:06:59.430387 24056 raft_consensus.cc:3037] T e75e025b0bb9463daf1fbffe11b17095 P 6acc4ef52963473d993cc66899bbe5cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:06:59.430426 24260 raft_consensus.cc:3037] T e75e025b0bb9463daf1fbffe11b17095 P df1ac1b971934f54a95716037b089d34 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:06:59.432672 24056 raft_consensus.cc:2454] T e75e025b0bb9463daf1fbffe11b17095 P 6acc4ef52963473d993cc66899bbe5cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a485f59cf414ba0bb3fedb581b344dd in term 1.
I20260808 13:06:59.432989 24260 raft_consensus.cc:2454] T e75e025b0bb9463daf1fbffe11b17095 P df1ac1b971934f54a95716037b089d34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a485f59cf414ba0bb3fedb581b344dd in term 1.
I20260808 13:06:59.433195 23760 leader_election.cc:302] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a485f59cf414ba0bb3fedb581b344dd, 6acc4ef52963473d993cc66899bbe5cb; no voters:
I20260808 13:06:59.433527 24366 raft_consensus.cc:2781] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:06:59.433969 24366 raft_consensus.cc:686] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd [term 1 LEADER]: Becoming Leader. State: Replica: 4a485f59cf414ba0bb3fedb581b344dd, State: Running, Role: LEADER
I20260808 13:06:59.434526 24366 consensus_queue.cc:237] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a485f59cf414ba0bb3fedb581b344dd" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35215 } } peers { permanent_uuid: "6acc4ef52963473d993cc66899bbe5cb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37813 } } peers { permanent_uuid: "df1ac1b971934f54a95716037b089d34" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44529 } }
I20260808 13:06:59.439432 23649 catalog_manager.cc:5495] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a485f59cf414ba0bb3fedb581b344dd (127.1.124.129). New cstate: current_term: 1 leader_uuid: "4a485f59cf414ba0bb3fedb581b344dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a485f59cf414ba0bb3fedb581b344dd" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6acc4ef52963473d993cc66899bbe5cb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37813 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df1ac1b971934f54a95716037b089d34" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44529 } health_report { overall_health: UNKNOWN } } }
W20260808 13:06:59.471088 24330 tablet.cc:2369] T e75e025b0bb9463daf1fbffe11b17095 P df1ac1b971934f54a95716037b089d34: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:06:59.485198 24350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:59.479338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49924 (local address 127.1.124.129:35215)
0808 13:06:59.479601 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:59.479607 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:59.479628 (+ 21us) server_negotiation.cc:407] Connection header received
0808 13:06:59.479880 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:59.479884 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:59.479986 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:59.480213 (+ 227us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:06:59.480609 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.481329 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.482835 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.483251 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.483394 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:59.483968 (+ 574us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:06:59.484572 (+ 604us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:06:59.484735 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:59.484836 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 13:06:59.484920 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 13:06:59.493393 24056 raft_consensus.cc:1261] T e75e025b0bb9463daf1fbffe11b17095 P 6acc4ef52963473d993cc66899bbe5cb [term 1 FOLLOWER]: Refusing update from remote peer 4a485f59cf414ba0bb3fedb581b344dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:06:59.493393 24260 raft_consensus.cc:1261] T e75e025b0bb9463daf1fbffe11b17095 P df1ac1b971934f54a95716037b089d34 [term 1 FOLLOWER]: Refusing update from remote peer 4a485f59cf414ba0bb3fedb581b344dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:06:59.494597 24366 consensus_queue.cc:1035] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6acc4ef52963473d993cc66899bbe5cb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:06:59.494952 24373 consensus_queue.cc:1035] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "df1ac1b971934f54a95716037b089d34" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:06:59.521901 23649 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:06:59.522235 23649 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54940:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:06:59.526428 23649 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a: Sending DeleteTablet for 3 replicas of tablet e75e025b0bb9463daf1fbffe11b17095
I20260808 13:06:59.527663 23814 tablet_service.cc:1514] Processing DeleteTablet for tablet e75e025b0bb9463daf1fbffe11b17095 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:06:59 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:57708
I20260808 13:06:59.527727 24028 tablet_service.cc:1514] Processing DeleteTablet for tablet e75e025b0bb9463daf1fbffe11b17095 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:06:59 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:34886
I20260808 13:06:59.528002 24230 tablet_service.cc:1514] Processing DeleteTablet for tablet e75e025b0bb9463daf1fbffe11b17095 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:06:59 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:57960
I20260808 13:06:59.528828 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 23735
I20260808 13:06:59.543262 24396 tablet_replica.cc:330] T e75e025b0bb9463daf1fbffe11b17095 P df1ac1b971934f54a95716037b089d34: stopping tablet replica
I20260808 13:06:59.544423 24395 tablet_replica.cc:330] T e75e025b0bb9463daf1fbffe11b17095 P 6acc4ef52963473d993cc66899bbe5cb: stopping tablet replica
I20260808 13:06:59.544930 24395 raft_consensus.cc:2229] T e75e025b0bb9463daf1fbffe11b17095 P 6acc4ef52963473d993cc66899bbe5cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:06:59.545282 24396 raft_consensus.cc:2229] T e75e025b0bb9463daf1fbffe11b17095 P df1ac1b971934f54a95716037b089d34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:06:59.545553 24395 raft_consensus.cc:2258] T e75e025b0bb9463daf1fbffe11b17095 P 6acc4ef52963473d993cc66899bbe5cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:06:59.546422 24394 tablet_replica.cc:330] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd: stopping tablet replica
I20260808 13:06:59.546896 24394 raft_consensus.cc:2229] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:06:59.547797 24394 raft_consensus.cc:2258] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:06:59.548324 24395 ts_tablet_manager.cc:1905] T e75e025b0bb9463daf1fbffe11b17095 P 6acc4ef52963473d993cc66899bbe5cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:06:59.548503 24396 raft_consensus.cc:2258] T e75e025b0bb9463daf1fbffe11b17095 P df1ac1b971934f54a95716037b089d34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:06:59.550535 24396 ts_tablet_manager.cc:1905] T e75e025b0bb9463daf1fbffe11b17095 P df1ac1b971934f54a95716037b089d34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:06:59.552877 24394 ts_tablet_manager.cc:1905] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:06:59.556013 24395 ts_tablet_manager.cc:1918] T e75e025b0bb9463daf1fbffe11b17095 P 6acc4ef52963473d993cc66899bbe5cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:06:59.556329 24395 log.cc:1196] T e75e025b0bb9463daf1fbffe11b17095 P 6acc4ef52963473d993cc66899bbe5cb: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/e75e025b0bb9463daf1fbffe11b17095
I20260808 13:06:59.557019 24395 ts_tablet_manager.cc:1939] T e75e025b0bb9463daf1fbffe11b17095 P 6acc4ef52963473d993cc66899bbe5cb: Deleting consensus metadata
I20260808 13:06:59.558290 24396 ts_tablet_manager.cc:1918] T e75e025b0bb9463daf1fbffe11b17095 P df1ac1b971934f54a95716037b089d34: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:06:59.558511 24396 log.cc:1196] T e75e025b0bb9463daf1fbffe11b17095 P df1ac1b971934f54a95716037b089d34: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/e75e025b0bb9463daf1fbffe11b17095
I20260808 13:06:59.559125 24396 ts_tablet_manager.cc:1939] T e75e025b0bb9463daf1fbffe11b17095 P df1ac1b971934f54a95716037b089d34: Deleting consensus metadata
I20260808 13:06:59.559374 23627 catalog_manager.cc:4841] TS 6acc4ef52963473d993cc66899bbe5cb (127.1.124.130:37813): tablet e75e025b0bb9463daf1fbffe11b17095 (table test-table [id=470bb72a9fbf4aa7859e57a7d3a12208]) successfully deleted
I20260808 13:06:59.560683 24394 ts_tablet_manager.cc:1918] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:06:59.560940 24394 log.cc:1196] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/e75e025b0bb9463daf1fbffe11b17095
I20260808 13:06:59.561072 23628 catalog_manager.cc:4841] TS df1ac1b971934f54a95716037b089d34 (127.1.124.131:44529): tablet e75e025b0bb9463daf1fbffe11b17095 (table test-table [id=470bb72a9fbf4aa7859e57a7d3a12208]) successfully deleted
I20260808 13:06:59.561671 24394 ts_tablet_manager.cc:1939] T e75e025b0bb9463daf1fbffe11b17095 P 4a485f59cf414ba0bb3fedb581b344dd: Deleting consensus metadata
Aug 08 13:06:59 dist-test-slave-t3q3 krb5kdc[23582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.129@KRBTEST.COM
I20260808 13:06:59.563467 23626 catalog_manager.cc:4841] TS 4a485f59cf414ba0bb3fedb581b344dd (127.1.124.129:35215): tablet e75e025b0bb9463daf1fbffe11b17095 (table test-table [id=470bb72a9fbf4aa7859e57a7d3a12208]) successfully deleted
I20260808 13:06:59.571998 24350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:59.530057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49928 (local address 127.1.124.129:35215)
0808 13:06:59.530249 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:59.530254 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:59.543583 (+ 13329us) server_negotiation.cc:407] Connection header received
0808 13:06:59.543836 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:59.543842 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:59.543931 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:59.544094 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:59.546376 (+ 2282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.547716 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.550965 (+ 3249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.551698 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.551802 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:59.564436 (+ 12634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:59.564451 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:59.564471 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:59.564537 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:59.568646 (+ 4109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:59.569418 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:59.569429 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:59.569433 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:59.569627 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:59.570139 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:59.570144 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:59.570148 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:59.570646 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:59.570958 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:59.571318 (+ 360us) server_negotiation.cc:299] Negotiation successful
0808 13:06:59.571587 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 13:06:59.572751 23904 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:06:59.685190 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 23735
I20260808 13:06:59.711855 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 23919
Aug 08 13:06:59 dist-test-slave-t3q3 krb5kdc[23582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.130@KRBTEST.COM
I20260808 13:06:59.728420 24352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:59.712791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54332 (local address 127.1.124.130:37813)
0808 13:06:59.713011 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:59.713017 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:59.713038 (+ 21us) server_negotiation.cc:407] Connection header received
0808 13:06:59.713165 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:59.713169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:59.713229 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:59.713339 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:59.714644 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.715416 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.716844 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.717264 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.717401 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:59.722244 (+ 4843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:59.722259 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:59.722275 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:59.722315 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:59.725351 (+ 3036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:59.726120 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:59.726127 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:59.726130 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:59.726257 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:59.726697 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:59.726702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:59.726705 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:59.727135 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:59.727458 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:59.727759 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 13:06:59.727992 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 13:06:59.729190 24126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:06:59.819540 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 23919
I20260808 13:06:59.846306 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 24140
Aug 08 13:06:59 dist-test-slave-t3q3 krb5kdc[23582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194416, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.131@KRBTEST.COM
I20260808 13:06:59.861966 24355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:06:59.847236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36270 (local address 127.1.124.131:44529)
0808 13:06:59.847418 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 13:06:59.847423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:06:59.847441 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:06:59.847598 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:06:59.847610 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:06:59.847677 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:06:59.847761 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:06:59.849189 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.849909 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.851302 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:06:59.851698 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:06:59.851825 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:06:59.856287 (+ 4462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:06:59.856303 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:06:59.856320 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:06:59.856361 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:06:59.859160 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:59.859772 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:59.859780 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:59.859782 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:59.859917 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:06:59.860438 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:06:59.860442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:06:59.860444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:06:59.860821 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:06:59.861091 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0808 13:06:59.861392 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 13:06:59.861626 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 13:06:59.862666 24322 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:06:59.952282 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 24140
I20260808 13:06:59.976984 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 23605
I20260808 13:06:59.978296 23688 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:07:00.092455 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 23605
I20260808 13:07:00.120263 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:43305
--webserver_interface=127.1.124.190
--webserver_port=45395
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:35975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:43305
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:07:00.470420 24404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:07:00.470752 24404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:07:00.470927 24404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:07:00.478349 24404 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:07:00.478472 24404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:07:00.478574 24404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:07:00.478667 24404 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:07:00.478780 24404 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:07:00.489363 24404 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:35975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:43305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:43305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=45395
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:07:00.490816 24404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:07:00.492270 24404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:07:00.520975 24417 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:00.521111 24413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:00.521039 24415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:07:00.521867 24404 server_base.cc:1034] running on GCE node
I20260808 13:07:00.522806 24404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:07:00.524657 24404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:07:00.525897 24404 hybrid_clock.cc:648] HybridClock initialized: now 1786194420525823 us; error 69 us; skew 500 ppm
I20260808 13:07:00.526345 24404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:07:00 dist-test-slave-t3q3 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194420, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:07:00.530133 24404 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:07:00.531832 24404 webserver.cc:460] Webserver started at http://127.1.124.190:45395/ using document root <none> and password file <none>
I20260808 13:07:00.532691 24404 fs_manager.cc:359] Metadata directory not provided
I20260808 13:07:00.532819 24404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:07:00.539245 24404 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260808 13:07:00.542666 24429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:00.544468 24404 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260808 13:07:00.544687 24404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "791268c5f57843bba2907ebd3433e77a"
format_stamp: "Formatted at 2026-08-08 13:06:57 on dist-test-slave-t3q3"
server_key: "ee19eb0b9185b9b68ded49d0f001b21e"
server_key_iv: "6b3f6dc453ccf0e91da6e5259702a0ff"
server_key_version: "encryptionkey@0"
I20260808 13:07:00.545846 24404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:07:00.564394 24404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:07:00.569285 24404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:07:00.569613 24404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:07:00.590106 24404 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:43305
I20260808 13:07:00.590168 24501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:43305 every 8 connection(s)
I20260808 13:07:00.591706 24404 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:07:00.598325 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 24404
I20260808 13:07:00.599082 24502 sys_catalog.cc:263] Verifying existing consensus state
I20260808 13:07:00.599242 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:35215
--local_ip_for_outbound_sockets=127.1.124.129
--tserver_master_addrs=127.1.124.190:43305
--webserver_port=38565
--webserver_interface=127.1.124.129
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:35975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:07:00.603102 24502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a: Bootstrap starting.
I20260808 13:07:00.623783 24502 log.cc:824] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a: Log is configured to *not* fsync() on all Append() calls
I20260808 13:07:00.636957 24502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 13:07:00.637537 24502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a: Bootstrap complete.
I20260808 13:07:00.647177 24502 raft_consensus.cc:348] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791268c5f57843bba2907ebd3433e77a" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43305 } }
I20260808 13:07:00.648066 24502 raft_consensus.cc:729] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 791268c5f57843bba2907ebd3433e77a, State: Initialized, Role: FOLLOWER
I20260808 13:07:00.648753 24502 consensus_queue.cc:260] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791268c5f57843bba2907ebd3433e77a" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43305 } }
I20260808 13:07:00.649011 24502 raft_consensus.cc:388] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:07:00.649145 24502 raft_consensus.cc:482] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:07:00.649349 24502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:07:00.653745 24502 raft_consensus.cc:504] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791268c5f57843bba2907ebd3433e77a" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43305 } }
I20260808 13:07:00.654322 24502 leader_election.cc:302] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 791268c5f57843bba2907ebd3433e77a; no voters:
I20260808 13:07:00.655040 24502 leader_election.cc:288] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 13:07:00.655200 24511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 2 FOLLOWER]: Leader election won for term 2
I20260808 13:07:00.656751 24511 raft_consensus.cc:686] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 2 LEADER]: Becoming Leader. State: Replica: 791268c5f57843bba2907ebd3433e77a, State: Running, Role: LEADER
I20260808 13:07:00.657383 24511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791268c5f57843bba2907ebd3433e77a" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43305 } }
I20260808 13:07:00.657780 24502 sys_catalog.cc:564] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:07:00.660126 24517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 791268c5f57843bba2907ebd3433e77a. Latest consensus state: current_term: 2 leader_uuid: "791268c5f57843bba2907ebd3433e77a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791268c5f57843bba2907ebd3433e77a" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43305 } } }
I20260808 13:07:00.660530 24517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [sys.catalog]: This master's current role is: LEADER
I20260808 13:07:00.661187 24513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "791268c5f57843bba2907ebd3433e77a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791268c5f57843bba2907ebd3433e77a" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43305 } } }
I20260808 13:07:00.661466 24513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [sys.catalog]: This master's current role is: LEADER
I20260808 13:07:00.666700 24523 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:07:00.676964 24523 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:07:00.678964 24523 catalog_manager.cc:1259] Loaded cluster ID: 758e54337868452d9506f08c2a124c67
I20260808 13:07:00.679111 24523 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:07:00.683751 24523 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 13:07:00.709004 24523 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 13:07:00.739603 24523 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:07:00.741541 24523 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a: Loaded TSK: 0
I20260808 13:07:00.743095 24523 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 13:07:00.983439 24507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:07:00.983744 24507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:07:00.983887 24507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:07:00.992375 24507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:07:00.992492 24507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:07:00.992666 24507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:07:01.002895 24507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:35975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:35215
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=38565
--tserver_master_addrs=127.1.124.190:43305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:07:01.004343 24507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:07:01.005681 24507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:07:01.021389 24554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:07:01.022962 24507 server_base.cc:1034] running on GCE node
W20260808 13:07:01.023065 24556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:01.023033 24553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:07:01.023922 24507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:07:01.024999 24507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:07:01.026237 24507 hybrid_clock.cc:648] HybridClock initialized: now 1786194421026190 us; error 35 us; skew 500 ppm
I20260808 13:07:01.026582 24507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:07:01 dist-test-slave-t3q3 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194421, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:07:01.030237 24507 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:07:01.031942 24507 webserver.cc:460] Webserver started at http://127.1.124.129:38565/ using document root <none> and password file <none>
I20260808 13:07:01.032729 24507 fs_manager.cc:359] Metadata directory not provided
I20260808 13:07:01.032902 24507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:07:01.038679 24507 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 13:07:01.041772 24567 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:01.043372 24507 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:07:01.043602 24507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "4a485f59cf414ba0bb3fedb581b344dd"
format_stamp: "Formatted at 2026-08-08 13:06:58 on dist-test-slave-t3q3"
server_key: "45a6a534451f30fb91ee2eec3f4ba46f"
server_key_iv: "3f4e95820fcfb25c5dcfad2283cccb71"
server_key_version: "encryptionkey@0"
I20260808 13:07:01.044886 24507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:07:01.070724 24507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:07:01.075608 24507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:07:01.075934 24507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:07:01.077073 24507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:07:01.078572 24507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:07:01.078668 24507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:01.078845 24507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:07:01.078929 24507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:01.112159 24507 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:35215
I20260808 13:07:01.112332 24763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:35215 every 8 connection(s)
I20260808 13:07:01.113791 24507 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:07:01.123914 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 24507
I20260808 13:07:01.124643 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:37813
--local_ip_for_outbound_sockets=127.1.124.130
--tserver_master_addrs=127.1.124.190:43305
--webserver_port=36307
--webserver_interface=127.1.124.130
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:35975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 13:07:01 dist-test-slave-t3q3 krb5kdc[23582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194421, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:07:01.141383 24768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:01.117285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.129:36403)
0808 13:07:01.118066 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:01.118107 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:01.119809 (+ 1702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:01.121106 (+ 1297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:01.121113 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:01.121630 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:07:01.122157 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:01.122174 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:01.124214 (+ 2040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:01.124226 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:01.125602 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:01.125615 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:01.127228 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:01.127234 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:01.127438 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:01.128310 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:07:01.128334 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:07:01.130976 (+ 2642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:07:01.135949 (+ 4973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:07:01.135963 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:07:01.135982 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:07:01.136750 (+ 768us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:07:01.137247 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:07:01.137253 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:07:01.137258 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:07:01.137536 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:07:01.138615 (+ 1079us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:07:01.138621 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:07:01.138982 (+ 361us) client_negotiation.cc:769] Sending connection context
0808 13:07:01.139313 (+ 331us) client_negotiation.cc:240] Negotiation successful
0808 13:07:01.139923 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":322,"threads_started":1}
I20260808 13:07:01.141383 24766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:01.116977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:36403 (local address 127.1.124.190:43305)
0808 13:07:01.117673 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:01.117694 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:01.118252 (+ 558us) server_negotiation.cc:407] Connection header received
0808 13:07:01.120334 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:01.120346 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:01.120659 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:01.120942 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:01.122359 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:01.123996 (+ 1637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:01.125784 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:01.126963 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:01.127186 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:01.131318 (+ 4132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:01.131332 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:01.131357 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:01.131419 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:01.135736 (+ 4317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:01.136898 (+ 1162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:01.136906 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:01.136911 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:01.137079 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:01.137676 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:01.137683 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:01.137689 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:01.138426 (+ 737us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:01.138625 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:01.139539 (+ 914us) server_negotiation.cc:299] Negotiation successful
0808 13:07:01.139955 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":255,"threads_started":1}
I20260808 13:07:01.144733 24764 heartbeater.cc:347] Connected to a master server at 127.1.124.190:43305
I20260808 13:07:01.145112 24764 heartbeater.cc:464] Registering TS with master...
I20260808 13:07:01.145848 24764 heartbeater.cc:511] Master 127.1.124.190:43305 requested a full tablet report, sending...
I20260808 13:07:01.148387 24449 ts_manager.cc:194] Registered new tserver with Master: 4a485f59cf414ba0bb3fedb581b344dd (127.1.124.129:35215)
I20260808 13:07:01.150754 24449 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:36403
W20260808 13:07:01.477680 24769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:07:01.477979 24769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:07:01.478077 24769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:07:01.485373 24769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:07:01.485497 24769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:07:01.485611 24769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:07:01.495909 24769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:35975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:37813
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=36307
--tserver_master_addrs=127.1.124.190:43305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:07:01.497390 24769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:07:01.498684 24769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:07:01.512409 24777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:01.512328 24779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:01.512367 24781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:07:01.513289 24769 server_base.cc:1034] running on GCE node
I20260808 13:07:01.513917 24769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:07:01.514928 24769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:07:01.516184 24769 hybrid_clock.cc:648] HybridClock initialized: now 1786194421516129 us; error 67 us; skew 500 ppm
I20260808 13:07:01.516535 24769 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:07:01 dist-test-slave-t3q3 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194421, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:07:01.519816 24769 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:07:01.521427 24769 webserver.cc:460] Webserver started at http://127.1.124.130:36307/ using document root <none> and password file <none>
I20260808 13:07:01.522146 24769 fs_manager.cc:359] Metadata directory not provided
I20260808 13:07:01.522265 24769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:07:01.527915 24769 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 13:07:01.530809 24792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:01.532256 24769 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:07:01.532399 24769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "6acc4ef52963473d993cc66899bbe5cb"
format_stamp: "Formatted at 2026-08-08 13:06:58 on dist-test-slave-t3q3"
server_key: "ca6c479dc37a2c95cc652c35527a960d"
server_key_iv: "3f2df92286f2ebba14fb05ff034a34c3"
server_key_version: "encryptionkey@0"
I20260808 13:07:01.533399 24769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:07:01.567477 24769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:07:01.571964 24769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:07:01.572232 24769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:07:01.573238 24769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:07:01.574520 24769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:07:01.574609 24769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:01.574692 24769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:07:01.574791 24769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:01.610559 24769 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:37813
I20260808 13:07:01.610687 24957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:37813 every 8 connection(s)
I20260808 13:07:01.612044 24769 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:07:01.619055 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 24769
I20260808 13:07:01.619720 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:44529
--local_ip_for_outbound_sockets=127.1.124.131
--tserver_master_addrs=127.1.124.190:43305
--webserver_port=42503
--webserver_interface=127.1.124.131
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:35975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 13:07:01 dist-test-slave-t3q3 krb5kdc[23582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194421, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:07:01.634578 24766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:01.614989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:45087 (local address 127.1.124.190:43305)
0808 13:07:01.615214 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:01.615220 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:01.616508 (+ 1288us) server_negotiation.cc:407] Connection header received
0808 13:07:01.618313 (+ 1805us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:01.618317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:01.618404 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:01.618498 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:01.620132 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:01.621306 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:01.622895 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:01.623681 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:01.623832 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:01.627295 (+ 3463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:01.627304 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:01.627311 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:01.627359 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:01.631011 (+ 3652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:01.632042 (+ 1031us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:01.632046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:01.632048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:01.632272 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:01.632786 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:01.632790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:01.632792 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:01.633080 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:01.633215 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:01.634115 (+ 900us) server_negotiation.cc:299] Negotiation successful
0808 13:07:01.634316 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 13:07:01.636250 24962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:01.615493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.130:45087)
0808 13:07:01.616328 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:01.616378 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:01.618021 (+ 1643us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:01.618716 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:01.618724 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:01.619299 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:07:01.619912 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:01.619938 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:01.621496 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:01.621500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:01.622741 (+ 1241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:01.622753 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:01.623891 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:01.623895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:01.624188 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:01.624881 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:07:01.624902 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:07:01.627057 (+ 2155us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:07:01.631203 (+ 4146us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:07:01.631216 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:07:01.631237 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:07:01.631899 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:07:01.632386 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:07:01.632393 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:07:01.632396 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:07:01.632674 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:07:01.633229 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:07:01.633239 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:07:01.633703 (+ 464us) client_negotiation.cc:769] Sending connection context
0808 13:07:01.634037 (+ 334us) client_negotiation.cc:240] Negotiation successful
0808 13:07:01.634641 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":286,"threads_started":1}
I20260808 13:07:01.637866 24958 heartbeater.cc:347] Connected to a master server at 127.1.124.190:43305
I20260808 13:07:01.638197 24958 heartbeater.cc:464] Registering TS with master...
I20260808 13:07:01.638952 24958 heartbeater.cc:511] Master 127.1.124.190:43305 requested a full tablet report, sending...
I20260808 13:07:01.640658 24449 ts_manager.cc:194] Registered new tserver with Master: 6acc4ef52963473d993cc66899bbe5cb (127.1.124.130:37813)
I20260808 13:07:01.641510 24449 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:45087
W20260808 13:07:01.973991 24963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:07:01.974416 24963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:07:01.974601 24963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:07:01.982125 24963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:07:01.982287 24963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:07:01.982506 24963 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:07:01.993098 24963 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:35975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:44529
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=42503
--tserver_master_addrs=127.1.124.190:43305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:07:01.994874 24963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:07:01.996351 24963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:07:02.010121 24973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:02.010217 24975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:02.010187 24977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:07:02.011538 24963 server_base.cc:1034] running on GCE node
I20260808 13:07:02.012213 24963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:07:02.013199 24963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:07:02.014434 24963 hybrid_clock.cc:648] HybridClock initialized: now 1786194422014383 us; error 50 us; skew 500 ppm
I20260808 13:07:02.014843 24963 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:07:02 dist-test-slave-t3q3 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194422, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:07:02.018162 24963 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:07:02.019709 24963 webserver.cc:460] Webserver started at http://127.1.124.131:42503/ using document root <none> and password file <none>
I20260808 13:07:02.020510 24963 fs_manager.cc:359] Metadata directory not provided
I20260808 13:07:02.020682 24963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:07:02.026137 24963 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.005s
I20260808 13:07:02.028911 24989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:02.030221 24963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:07:02.030397 24963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "df1ac1b971934f54a95716037b089d34"
format_stamp: "Formatted at 2026-08-08 13:06:59 on dist-test-slave-t3q3"
server_key: "28463d66a8ebdeb2df6d4f46434dc890"
server_key_iv: "bedac146c41773fde0c222688632ba23"
server_key_version: "encryptionkey@0"
I20260808 13:07:02.031373 24963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:07:02.047130 24963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:07:02.051926 24963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:07:02.052261 24963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:07:02.053287 24963 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:07:02.054638 24963 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:07:02.054764 24963 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:02.054901 24963 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:07:02.054996 24963 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:02.089993 24963 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:44529
I20260808 13:07:02.090083 25146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:44529 every 8 connection(s)
I20260808 13:07:02.091547 24963 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:07:02.094406 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 24963
Aug 08 13:07:02 dist-test-slave-t3q3 krb5kdc[23582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194422, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:07:02.117784 25154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:02.095380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59928 (local address 127.1.124.190:43305)
0808 13:07:02.100048 (+ 4668us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:02.100052 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:02.100111 (+ 59us) server_negotiation.cc:407] Connection header received
0808 13:07:02.100164 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:02.100169 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:02.100237 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:02.100323 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:02.102424 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.103524 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.105604 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.106257 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.106412 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:02.110123 (+ 3711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:02.110128 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:02.110131 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:02.110165 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:02.113661 (+ 3496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:02.114625 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:02.114631 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:02.114635 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:02.114810 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:02.115911 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:02.115915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:02.115918 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:02.116363 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:02.116648 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:02.117020 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 13:07:02.117260 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":4573,"thread_start_us":4466,"threads_started":1}
I20260808 13:07:02.119163 24766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:02.094304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:35219 (local address 127.1.124.190:43305)
0808 13:07:02.094526 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:02.094532 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:02.095990 (+ 1458us) server_negotiation.cc:407] Connection header received
0808 13:07:02.099749 (+ 3759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:02.099753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:02.099860 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:02.099990 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:02.102267 (+ 2277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.103552 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.105919 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.106643 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.106782 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:02.111584 (+ 4802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:02.111591 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:02.111595 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:02.111678 (+ 83us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:02.115138 (+ 3460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:02.115881 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:02.115888 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:02.115895 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:02.116108 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:02.116657 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:02.116663 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:02.116666 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:02.117012 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:02.117162 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:02.118618 (+ 1456us) server_negotiation.cc:299] Negotiation successful
0808 13:07:02.118830 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":80}
I20260808 13:07:02.120162 25149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:02.094812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.131:35219)
0808 13:07:02.095770 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:02.095843 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:02.099228 (+ 3385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:02.100314 (+ 1086us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:02.100329 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:02.101167 (+ 838us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:07:02.102043 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:02.102073 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.103708 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.103713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:02.105766 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:02.105780 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.106898 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.106908 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:02.107264 (+ 356us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:02.107958 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:07:02.107994 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:07:02.111259 (+ 3265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:07:02.115282 (+ 4023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:07:02.115292 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:07:02.115309 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:07:02.115747 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:07:02.116258 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:07:02.116263 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:07:02.116268 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:07:02.116509 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:07:02.117568 (+ 1059us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:07:02.117578 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:07:02.117993 (+ 415us) client_negotiation.cc:769] Sending connection context
0808 13:07:02.118300 (+ 307us) client_negotiation.cc:240] Negotiation successful
0808 13:07:02.118846 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":379,"threads_started":1}
I20260808 13:07:02.121551 25147 heartbeater.cc:347] Connected to a master server at 127.1.124.190:43305
I20260808 13:07:02.121858 25147 heartbeater.cc:464] Registering TS with master...
I20260808 13:07:02.122592 25147 heartbeater.cc:511] Master 127.1.124.190:43305 requested a full tablet report, sending...
I20260808 13:07:02.123999 24449 ts_manager.cc:194] Registered new tserver with Master: df1ac1b971934f54a95716037b089d34 (127.1.124.131:44529)
I20260808 13:07:02.124886 24449 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:35219
I20260808 13:07:02.132863 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:07:02.140547 24766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:02.134011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59932 (local address 127.1.124.190:43305)
0808 13:07:02.134249 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:02.134259 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:02.134627 (+ 368us) server_negotiation.cc:407] Connection header received
0808 13:07:02.134885 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:02.134888 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:02.135083 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:02.135244 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:07:02.135652 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.136575 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.138271 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.138722 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.138862 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:02.139401 (+ 539us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:07:02.139916 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:07:02.140142 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:02.140194 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 13:07:02.140280 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 13:07:02.141604 24449 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:59932:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:07:02.149132 24449 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:07:02.154300 24764 heartbeater.cc:503] Master 127.1.124.190:43305 was elected leader, sending a full tablet report...
I20260808 13:07:02.187443 25165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:02.162224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:35215 (local address 127.0.0.1:49934)
0808 13:07:02.173338 (+ 11114us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:02.173387 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:02.173573 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:02.179657 (+ 6084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:02.179664 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:02.179695 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:07:02.180274 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:02.180292 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.182127 (+ 1835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.182137 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:02.184950 (+ 2813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:02.184966 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.186316 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.186323 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:02.186529 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:02.186588 (+ 59us) client_negotiation.cc:769] Sending connection context
0808 13:07:02.186746 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 13:07:02.186934 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10833,"mutex_wait_us":252,"thread_start_us":17448,"threads_started":1}
I20260808 13:07:02.188232 25166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:02.162727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36284 (local address 127.1.124.131:44529)
0808 13:07:02.175964 (+ 13237us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:02.175977 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:02.178497 (+ 2520us) server_negotiation.cc:407] Connection header received
0808 13:07:02.179090 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:02.179113 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:02.179313 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:02.179482 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:07:02.180541 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.181808 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.185121 (+ 3313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.186545 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.186675 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:02.186911 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:02.187696 (+ 785us) server_negotiation.cc:299] Negotiation successful
0808 13:07:02.187869 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12874,"thread_start_us":13272,"threads_started":1}
I20260808 13:07:02.188408 25169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:02.164777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54348 (local address 127.1.124.130:37813)
0808 13:07:02.178936 (+ 14159us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:02.178946 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:02.179722 (+ 776us) server_negotiation.cc:407] Connection header received
0808 13:07:02.179940 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:02.179947 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:02.180189 (+ 242us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:02.180387 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:07:02.181125 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.182614 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.184895 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.186357 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.187105 (+ 748us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:02.187283 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:02.187882 (+ 599us) server_negotiation.cc:299] Negotiation successful
0808 13:07:02.188042 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14012,"thread_start_us":13898,"threads_started":1}
I20260808 13:07:02.188567 25164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:02.162490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49934 (local address 127.1.124.129:35215)
0808 13:07:02.178758 (+ 16268us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:02.178768 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:02.178796 (+ 28us) server_negotiation.cc:407] Connection header received
0808 13:07:02.178903 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:02.178911 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:02.179199 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:02.180190 (+ 991us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:07:02.180451 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.181903 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.185153 (+ 3250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.186101 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.186264 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:02.186525 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:02.187979 (+ 1454us) server_negotiation.cc:299] Negotiation successful
0808 13:07:02.188173 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16066,"thread_start_us":16860,"threads_started":1}
I20260808 13:07:02.187539 25167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:02.162568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:44529 (local address 127.0.0.1:36284)
0808 13:07:02.178355 (+ 15787us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:02.178381 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:02.178524 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:02.179821 (+ 1297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:02.179827 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:02.179854 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:07:02.180352 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:02.180361 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.181958 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.181965 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:02.184950 (+ 2985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:02.184966 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.186746 (+ 1780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.186752 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:02.186901 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:02.186930 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 13:07:02.187005 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 13:07:02.187149 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15643,"mutex_wait_us":214,"thread_start_us":10327,"threads_started":1}
I20260808 13:07:02.188408 25163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:02.164508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:37813 (local address 127.0.0.1:54348)
0808 13:07:02.179406 (+ 14898us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:02.179426 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:02.179545 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:02.180508 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:02.180513 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:02.180538 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:07:02.180927 (+ 389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:02.180940 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.182784 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.182788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:02.184729 (+ 1941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:02.184747 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.187465 (+ 2718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.187472 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:02.187657 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:02.187703 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 13:07:02.187816 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 13:07:02.187947 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14759,"mutex_wait_us":40,"thread_start_us":14101,"threads_started":1}
I20260808 13:07:02.192351 24652 tablet_service.cc:1467] Processing CreateTablet for tablet adbb8077c8af43b7bec00c80cff69350 (DEFAULT_TABLE table=test-table [id=36f2fa06f06e4b3884cc116b6c9c0cf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:07:02.192397 24859 tablet_service.cc:1467] Processing CreateTablet for tablet adbb8077c8af43b7bec00c80cff69350 (DEFAULT_TABLE table=test-table [id=36f2fa06f06e4b3884cc116b6c9c0cf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:07:02.192395 25059 tablet_service.cc:1467] Processing CreateTablet for tablet adbb8077c8af43b7bec00c80cff69350 (DEFAULT_TABLE table=test-table [id=36f2fa06f06e4b3884cc116b6c9c0cf5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:07:02.193692 24652 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adbb8077c8af43b7bec00c80cff69350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:07:02.193763 24859 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adbb8077c8af43b7bec00c80cff69350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:07:02.193761 25059 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet adbb8077c8af43b7bec00c80cff69350. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:07:02.206001 25170 tablet_bootstrap.cc:492] T adbb8077c8af43b7bec00c80cff69350 P 4a485f59cf414ba0bb3fedb581b344dd: Bootstrap starting.
I20260808 13:07:02.209589 25170 tablet_bootstrap.cc:654] T adbb8077c8af43b7bec00c80cff69350 P 4a485f59cf414ba0bb3fedb581b344dd: Neither blocks nor log segments found. Creating new log.
I20260808 13:07:02.211105 25170 log.cc:824] T adbb8077c8af43b7bec00c80cff69350 P 4a485f59cf414ba0bb3fedb581b344dd: Log is configured to *not* fsync() on all Append() calls
I20260808 13:07:02.211196 25172 tablet_bootstrap.cc:492] T adbb8077c8af43b7bec00c80cff69350 P 6acc4ef52963473d993cc66899bbe5cb: Bootstrap starting.
I20260808 13:07:02.212376 25175 tablet_bootstrap.cc:492] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34: Bootstrap starting.
I20260808 13:07:02.215473 25172 tablet_bootstrap.cc:654] T adbb8077c8af43b7bec00c80cff69350 P 6acc4ef52963473d993cc66899bbe5cb: Neither blocks nor log segments found. Creating new log.
I20260808 13:07:02.216171 25175 tablet_bootstrap.cc:654] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34: Neither blocks nor log segments found. Creating new log.
I20260808 13:07:02.216727 25170 tablet_bootstrap.cc:492] T adbb8077c8af43b7bec00c80cff69350 P 4a485f59cf414ba0bb3fedb581b344dd: No bootstrap required, opened a new log
I20260808 13:07:02.216866 25172 log.cc:824] T adbb8077c8af43b7bec00c80cff69350 P 6acc4ef52963473d993cc66899bbe5cb: Log is configured to *not* fsync() on all Append() calls
I20260808 13:07:02.217028 25170 ts_tablet_manager.cc:1397] T adbb8077c8af43b7bec00c80cff69350 P 4a485f59cf414ba0bb3fedb581b344dd: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.002s
I20260808 13:07:02.217294 25175 log.cc:824] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34: Log is configured to *not* fsync() on all Append() calls
I20260808 13:07:02.220343 25172 tablet_bootstrap.cc:492] T adbb8077c8af43b7bec00c80cff69350 P 6acc4ef52963473d993cc66899bbe5cb: No bootstrap required, opened a new log
I20260808 13:07:02.220645 25172 ts_tablet_manager.cc:1397] T adbb8077c8af43b7bec00c80cff69350 P 6acc4ef52963473d993cc66899bbe5cb: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260808 13:07:02.227660 25175 tablet_bootstrap.cc:492] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34: No bootstrap required, opened a new log
I20260808 13:07:02.227988 25175 ts_tablet_manager.cc:1397] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34: Time spent bootstrapping tablet: real 0.016s user 0.003s sys 0.010s
I20260808 13:07:02.228127 25170 raft_consensus.cc:348] T adbb8077c8af43b7bec00c80cff69350 P 4a485f59cf414ba0bb3fedb581b344dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a485f59cf414ba0bb3fedb581b344dd" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35215 } } peers { permanent_uuid: "df1ac1b971934f54a95716037b089d34" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44529 } } peers { permanent_uuid: "6acc4ef52963473d993cc66899bbe5cb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37813 } }
I20260808 13:07:02.228631 25170 raft_consensus.cc:374] T adbb8077c8af43b7bec00c80cff69350 P 4a485f59cf414ba0bb3fedb581b344dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:07:02.228783 25170 raft_consensus.cc:729] T adbb8077c8af43b7bec00c80cff69350 P 4a485f59cf414ba0bb3fedb581b344dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a485f59cf414ba0bb3fedb581b344dd, State: Initialized, Role: FOLLOWER
I20260808 13:07:02.229473 25170 consensus_queue.cc:260] T adbb8077c8af43b7bec00c80cff69350 P 4a485f59cf414ba0bb3fedb581b344dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a485f59cf414ba0bb3fedb581b344dd" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35215 } } peers { permanent_uuid: "df1ac1b971934f54a95716037b089d34" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44529 } } peers { permanent_uuid: "6acc4ef52963473d993cc66899bbe5cb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37813 } }
I20260808 13:07:02.231401 25170 ts_tablet_manager.cc:1428] T adbb8077c8af43b7bec00c80cff69350 P 4a485f59cf414ba0bb3fedb581b344dd: Time spent starting tablet: real 0.014s user 0.012s sys 0.002s
I20260808 13:07:02.231570 25172 raft_consensus.cc:348] T adbb8077c8af43b7bec00c80cff69350 P 6acc4ef52963473d993cc66899bbe5cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a485f59cf414ba0bb3fedb581b344dd" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35215 } } peers { permanent_uuid: "df1ac1b971934f54a95716037b089d34" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44529 } } peers { permanent_uuid: "6acc4ef52963473d993cc66899bbe5cb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37813 } }
I20260808 13:07:02.232101 25172 raft_consensus.cc:374] T adbb8077c8af43b7bec00c80cff69350 P 6acc4ef52963473d993cc66899bbe5cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:07:02.232230 25172 raft_consensus.cc:729] T adbb8077c8af43b7bec00c80cff69350 P 6acc4ef52963473d993cc66899bbe5cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6acc4ef52963473d993cc66899bbe5cb, State: Initialized, Role: FOLLOWER
I20260808 13:07:02.232816 25172 consensus_queue.cc:260] T adbb8077c8af43b7bec00c80cff69350 P 6acc4ef52963473d993cc66899bbe5cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a485f59cf414ba0bb3fedb581b344dd" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35215 } } peers { permanent_uuid: "df1ac1b971934f54a95716037b089d34" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44529 } } peers { permanent_uuid: "6acc4ef52963473d993cc66899bbe5cb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37813 } }
I20260808 13:07:02.233965 24958 heartbeater.cc:503] Master 127.1.124.190:43305 was elected leader, sending a full tablet report...
I20260808 13:07:02.234997 25172 ts_tablet_manager.cc:1428] T adbb8077c8af43b7bec00c80cff69350 P 6acc4ef52963473d993cc66899bbe5cb: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260808 13:07:02.238616 25175 raft_consensus.cc:348] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a485f59cf414ba0bb3fedb581b344dd" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35215 } } peers { permanent_uuid: "df1ac1b971934f54a95716037b089d34" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44529 } } peers { permanent_uuid: "6acc4ef52963473d993cc66899bbe5cb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37813 } }
I20260808 13:07:02.239068 25175 raft_consensus.cc:374] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:07:02.239151 25175 raft_consensus.cc:729] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df1ac1b971934f54a95716037b089d34, State: Initialized, Role: FOLLOWER
I20260808 13:07:02.239621 25175 consensus_queue.cc:260] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a485f59cf414ba0bb3fedb581b344dd" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35215 } } peers { permanent_uuid: "df1ac1b971934f54a95716037b089d34" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44529 } } peers { permanent_uuid: "6acc4ef52963473d993cc66899bbe5cb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37813 } }
I20260808 13:07:02.240671 25147 heartbeater.cc:503] Master 127.1.124.190:43305 was elected leader, sending a full tablet report...
I20260808 13:07:02.241183 25175 ts_tablet_manager.cc:1428] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34: Time spent starting tablet: real 0.013s user 0.010s sys 0.002s
I20260808 13:07:02.252466 25188 raft_consensus.cc:482] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:07:02.252767 25188 raft_consensus.cc:504] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a485f59cf414ba0bb3fedb581b344dd" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35215 } } peers { permanent_uuid: "df1ac1b971934f54a95716037b089d34" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44529 } } peers { permanent_uuid: "6acc4ef52963473d993cc66899bbe5cb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37813 } }
I20260808 13:07:02.254554 25188 leader_election.cc:288] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4a485f59cf414ba0bb3fedb581b344dd (127.1.124.129:35215), 6acc4ef52963473d993cc66899bbe5cb (127.1.124.130:37813)
I20260808 13:07:02.262348 25164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:02.255036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:36347 (local address 127.1.124.129:35215)
0808 13:07:02.255275 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:02.255282 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:02.255508 (+ 226us) server_negotiation.cc:407] Connection header received
0808 13:07:02.255754 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:02.255759 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:02.255839 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:02.255921 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:07:02.256437 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.257523 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.259971 (+ 2448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.261439 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.261610 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:02.261691 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:02.261937 (+ 246us) server_negotiation.cc:299] Negotiation successful
0808 13:07:02.262057 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 13:07:02.262522 25149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:02.255022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:35215 (local address 127.1.124.131:36347)
0808 13:07:02.255361 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:02.255400 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:02.255564 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:02.255975 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:02.255980 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:02.256032 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:07:02.256270 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:02.256280 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.257655 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.257658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:02.259768 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:02.259784 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.261589 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.261600 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:02.261745 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:02.261770 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 13:07:02.261904 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 13:07:02.262050 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":214,"mutex_wait_us":1,"spinlock_wait_cycles":20992}
I20260808 13:07:02.262863 25169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:02.254997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:46439 (local address 127.1.124.130:37813)
0808 13:07:02.255300 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:02.255307 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:02.255994 (+ 687us) server_negotiation.cc:407] Connection header received
0808 13:07:02.256209 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:02.256213 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:02.256270 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:02.256343 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:07:02.256872 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.257830 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.260375 (+ 2545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.261726 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.261847 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:02.261942 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:02.262328 (+ 386us) server_negotiation.cc:299] Negotiation successful
0808 13:07:02.262632 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139}
I20260808 13:07:02.262913 25189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:02.255094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:37813 (local address 127.1.124.131:46439)
0808 13:07:02.255857 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:02.255870 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:02.256036 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:02.256387 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:02.256392 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:02.256421 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:07:02.256713 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:02.256720 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.257963 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.257968 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:02.260005 (+ 2037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:02.260015 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.261928 (+ 1913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.261933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:02.262113 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:02.262144 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 13:07:02.262288 (+ 144us) client_negotiation.cc:240] Negotiation successful
0808 13:07:02.262411 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"spinlock_wait_cycles":25344,"thread_start_us":418,"threads_started":1}
I20260808 13:07:02.263845 24894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adbb8077c8af43b7bec00c80cff69350" candidate_uuid: "df1ac1b971934f54a95716037b089d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6acc4ef52963473d993cc66899bbe5cb" is_pre_election: true
I20260808 13:07:02.263845 24682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adbb8077c8af43b7bec00c80cff69350" candidate_uuid: "df1ac1b971934f54a95716037b089d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a485f59cf414ba0bb3fedb581b344dd" is_pre_election: true
I20260808 13:07:02.264366 24682 raft_consensus.cc:2454] T adbb8077c8af43b7bec00c80cff69350 P 4a485f59cf414ba0bb3fedb581b344dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df1ac1b971934f54a95716037b089d34 in term 0.
I20260808 13:07:02.264366 24894 raft_consensus.cc:2454] T adbb8077c8af43b7bec00c80cff69350 P 6acc4ef52963473d993cc66899bbe5cb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate df1ac1b971934f54a95716037b089d34 in term 0.
I20260808 13:07:02.265106 24992 leader_election.cc:302] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4a485f59cf414ba0bb3fedb581b344dd, 6acc4ef52963473d993cc66899bbe5cb, df1ac1b971934f54a95716037b089d34; no voters:
I20260808 13:07:02.265523 25188 raft_consensus.cc:2781] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:07:02.265688 25188 raft_consensus.cc:482] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:07:02.265811 25188 raft_consensus.cc:3037] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:07:02.268117 25188 raft_consensus.cc:504] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a485f59cf414ba0bb3fedb581b344dd" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35215 } } peers { permanent_uuid: "df1ac1b971934f54a95716037b089d34" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44529 } } peers { permanent_uuid: "6acc4ef52963473d993cc66899bbe5cb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37813 } }
I20260808 13:07:02.268864 25188 leader_election.cc:288] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34 [CANDIDATE]: Term 1 election: Requested vote from peers 4a485f59cf414ba0bb3fedb581b344dd (127.1.124.129:35215), 6acc4ef52963473d993cc66899bbe5cb (127.1.124.130:37813)
I20260808 13:07:02.269263 24682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adbb8077c8af43b7bec00c80cff69350" candidate_uuid: "df1ac1b971934f54a95716037b089d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4a485f59cf414ba0bb3fedb581b344dd"
I20260808 13:07:02.269346 24894 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "adbb8077c8af43b7bec00c80cff69350" candidate_uuid: "df1ac1b971934f54a95716037b089d34" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6acc4ef52963473d993cc66899bbe5cb"
I20260808 13:07:02.269461 24682 raft_consensus.cc:3037] T adbb8077c8af43b7bec00c80cff69350 P 4a485f59cf414ba0bb3fedb581b344dd [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:07:02.269603 24894 raft_consensus.cc:3037] T adbb8077c8af43b7bec00c80cff69350 P 6acc4ef52963473d993cc66899bbe5cb [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:07:02.271723 24682 raft_consensus.cc:2454] T adbb8077c8af43b7bec00c80cff69350 P 4a485f59cf414ba0bb3fedb581b344dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df1ac1b971934f54a95716037b089d34 in term 1.
I20260808 13:07:02.271842 24894 raft_consensus.cc:2454] T adbb8077c8af43b7bec00c80cff69350 P 6acc4ef52963473d993cc66899bbe5cb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate df1ac1b971934f54a95716037b089d34 in term 1.
I20260808 13:07:02.272136 24991 leader_election.cc:302] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a485f59cf414ba0bb3fedb581b344dd, df1ac1b971934f54a95716037b089d34; no voters:
I20260808 13:07:02.272439 25188 raft_consensus.cc:2781] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:07:02.272886 25188 raft_consensus.cc:686] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34 [term 1 LEADER]: Becoming Leader. State: Replica: df1ac1b971934f54a95716037b089d34, State: Running, Role: LEADER
I20260808 13:07:02.273430 25188 consensus_queue.cc:237] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a485f59cf414ba0bb3fedb581b344dd" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35215 } } peers { permanent_uuid: "df1ac1b971934f54a95716037b089d34" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44529 } } peers { permanent_uuid: "6acc4ef52963473d993cc66899bbe5cb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37813 } }
I20260808 13:07:02.277999 24449 catalog_manager.cc:5495] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34 reported cstate change: term changed from 0 to 1, leader changed from <none> to df1ac1b971934f54a95716037b089d34 (127.1.124.131). New cstate: current_term: 1 leader_uuid: "df1ac1b971934f54a95716037b089d34" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a485f59cf414ba0bb3fedb581b344dd" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35215 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "df1ac1b971934f54a95716037b089d34" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6acc4ef52963473d993cc66899bbe5cb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37813 } health_report { overall_health: UNKNOWN } } }
I20260808 13:07:02.298455 25166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:02.292448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36286 (local address 127.1.124.131:44529)
0808 13:07:02.292668 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:02.292675 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:02.292694 (+ 19us) server_negotiation.cc:407] Connection header received
0808 13:07:02.292957 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:02.292970 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:02.293048 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:02.293227 (+ 179us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:07:02.293589 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.294342 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.296011 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.296535 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.296687 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:02.297339 (+ 652us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:07:02.297855 (+ 516us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:07:02.298029 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:02.298096 (+ 67us) server_negotiation.cc:299] Negotiation successful
0808 13:07:02.298191 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 13:07:02.306910 24894 raft_consensus.cc:1261] T adbb8077c8af43b7bec00c80cff69350 P 6acc4ef52963473d993cc66899bbe5cb [term 1 FOLLOWER]: Refusing update from remote peer df1ac1b971934f54a95716037b089d34: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:07:02.306918 24682 raft_consensus.cc:1261] T adbb8077c8af43b7bec00c80cff69350 P 4a485f59cf414ba0bb3fedb581b344dd [term 1 FOLLOWER]: Refusing update from remote peer df1ac1b971934f54a95716037b089d34: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:07:02.308025 25188 consensus_queue.cc:1035] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6acc4ef52963473d993cc66899bbe5cb" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 37813 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:07:02.308411 25193 consensus_queue.cc:1035] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4a485f59cf414ba0bb3fedb581b344dd" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 35215 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:07:02.322106 25199 mvcc.cc:205] Tried to move back new op lower bound from 7316252353759006720 to 7316252353636032512. Current Snapshot: MvccSnapshot[applied={T|T < 7316252353759006720}]
I20260808 13:07:02.338007 24449 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:59932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:07:02.338299 24449 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:59932:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:07:02.342123 24449 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a: Sending DeleteTablet for 3 replicas of tablet adbb8077c8af43b7bec00c80cff69350
I20260808 13:07:02.343341 24652 tablet_service.cc:1514] Processing DeleteTablet for tablet adbb8077c8af43b7bec00c80cff69350 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:07:02 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:49934
I20260808 13:07:02.343406 25059 tablet_service.cc:1514] Processing DeleteTablet for tablet adbb8077c8af43b7bec00c80cff69350 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:07:02 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:36284
I20260808 13:07:02.344413 24859 tablet_service.cc:1514] Processing DeleteTablet for tablet adbb8077c8af43b7bec00c80cff69350 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:07:02 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:54348
I20260808 13:07:02.346551 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 24507
I20260808 13:07:02.355746 25208 tablet_replica.cc:330] T adbb8077c8af43b7bec00c80cff69350 P 4a485f59cf414ba0bb3fedb581b344dd: stopping tablet replica
I20260808 13:07:02.356961 25208 raft_consensus.cc:2229] T adbb8077c8af43b7bec00c80cff69350 P 4a485f59cf414ba0bb3fedb581b344dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:07:02.358331 25208 raft_consensus.cc:2258] T adbb8077c8af43b7bec00c80cff69350 P 4a485f59cf414ba0bb3fedb581b344dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:07:02.362440 25209 tablet_replica.cc:330] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34: stopping tablet replica
I20260808 13:07:02.363261 25208 ts_tablet_manager.cc:1905] T adbb8077c8af43b7bec00c80cff69350 P 4a485f59cf414ba0bb3fedb581b344dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:07:02.363799 25209 raft_consensus.cc:2229] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:07:02.366212 25211 tablet_replica.cc:330] T adbb8077c8af43b7bec00c80cff69350 P 6acc4ef52963473d993cc66899bbe5cb: stopping tablet replica
I20260808 13:07:02.366671 25211 raft_consensus.cc:2229] T adbb8077c8af43b7bec00c80cff69350 P 6acc4ef52963473d993cc66899bbe5cb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:07:02.367065 25209 raft_consensus.cc:2258] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:07:02.367206 25211 raft_consensus.cc:2258] T adbb8077c8af43b7bec00c80cff69350 P 6acc4ef52963473d993cc66899bbe5cb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:07:02.369665 25209 ts_tablet_manager.cc:1905] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:07:02.371807 25208 ts_tablet_manager.cc:1918] T adbb8077c8af43b7bec00c80cff69350 P 4a485f59cf414ba0bb3fedb581b344dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:07:02.372017 25208 log.cc:1196] T adbb8077c8af43b7bec00c80cff69350 P 4a485f59cf414ba0bb3fedb581b344dd: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/adbb8077c8af43b7bec00c80cff69350
I20260808 13:07:02.372612 25208 ts_tablet_manager.cc:1939] T adbb8077c8af43b7bec00c80cff69350 P 4a485f59cf414ba0bb3fedb581b344dd: Deleting consensus metadata
I20260808 13:07:02.374722 24431 catalog_manager.cc:4841] TS 4a485f59cf414ba0bb3fedb581b344dd (127.1.124.129:35215): tablet adbb8077c8af43b7bec00c80cff69350 (table test-table [id=36f2fa06f06e4b3884cc116b6c9c0cf5]) successfully deleted
I20260808 13:07:02.377012 25211 ts_tablet_manager.cc:1905] T adbb8077c8af43b7bec00c80cff69350 P 6acc4ef52963473d993cc66899bbe5cb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:07:02.377074 25164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:02.348555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49940 (local address 127.1.124.129:35215)
0808 13:07:02.348810 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:02.348819 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:02.348843 (+ 24us) server_negotiation.cc:407] Connection header received
0808 13:07:02.348919 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:02.348928 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:02.349052 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:02.349174 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:02.352362 (+ 3188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.353996 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.356674 (+ 2678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.357651 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.357805 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:02.363739 (+ 5934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:02.363760 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:02.363784 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:02.363844 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:02.368281 (+ 4437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:02.373318 (+ 5037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:02.373327 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:02.373331 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:02.373517 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:02.374062 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:02.374066 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:02.374069 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:02.374716 (+ 647us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:02.375115 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:02.376444 (+ 1329us) server_negotiation.cc:299] Negotiation successful
0808 13:07:02.376711 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260808 13:07:02.377184 25209 ts_tablet_manager.cc:1918] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:07:02.377373 25209 log.cc:1196] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/adbb8077c8af43b7bec00c80cff69350
I20260808 13:07:02.377959 25209 ts_tablet_manager.cc:1939] T adbb8077c8af43b7bec00c80cff69350 P df1ac1b971934f54a95716037b089d34: Deleting consensus metadata
I20260808 13:07:02.378249 24754 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:07:02.379726 24433 catalog_manager.cc:4841] TS df1ac1b971934f54a95716037b089d34 (127.1.124.131:44529): tablet adbb8077c8af43b7bec00c80cff69350 (table test-table [id=36f2fa06f06e4b3884cc116b6c9c0cf5]) successfully deleted
I20260808 13:07:02.386026 25211 ts_tablet_manager.cc:1918] T adbb8077c8af43b7bec00c80cff69350 P 6acc4ef52963473d993cc66899bbe5cb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:07:02.386235 25211 log.cc:1196] T adbb8077c8af43b7bec00c80cff69350 P 6acc4ef52963473d993cc66899bbe5cb: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/adbb8077c8af43b7bec00c80cff69350
I20260808 13:07:02.386911 25211 ts_tablet_manager.cc:1939] T adbb8077c8af43b7bec00c80cff69350 P 6acc4ef52963473d993cc66899bbe5cb: Deleting consensus metadata
I20260808 13:07:02.388942 24432 catalog_manager.cc:4841] TS 6acc4ef52963473d993cc66899bbe5cb (127.1.124.130:37813): tablet adbb8077c8af43b7bec00c80cff69350 (table test-table [id=36f2fa06f06e4b3884cc116b6c9c0cf5]) successfully deleted
I20260808 13:07:02.494550 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 24507
I20260808 13:07:02.523092 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 24769
I20260808 13:07:02.539443 25169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:02.523997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54360 (local address 127.1.124.130:37813)
0808 13:07:02.524181 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:02.524187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:02.524313 (+ 126us) server_negotiation.cc:407] Connection header received
0808 13:07:02.524527 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:02.524532 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:02.524613 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:02.524734 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:02.526137 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.526883 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.528382 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.529053 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.529203 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:02.531823 (+ 2620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:02.531842 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:02.531862 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:02.531922 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:02.535919 (+ 3997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:02.536818 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:02.536828 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:02.536832 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:02.536999 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:02.537460 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:02.537465 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:02.537470 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:02.537999 (+ 529us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:02.538322 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:02.538784 (+ 462us) server_negotiation.cc:299] Negotiation successful
0808 13:07:02.539033 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 13:07:02.540278 24953 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:07:02.631259 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 24769
I20260808 13:07:02.660583 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 24963
I20260808 13:07:02.676438 25166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:02.661458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36292 (local address 127.1.124.131:44529)
0808 13:07:02.661680 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:02.661687 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:02.661811 (+ 124us) server_negotiation.cc:407] Connection header received
0808 13:07:02.662053 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:02.662059 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:02.662140 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:02.662280 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:02.663674 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.664633 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.666048 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:02.666529 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:02.666669 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:02.669770 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:02.669783 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:02.669800 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:02.669837 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:02.673204 (+ 3367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:02.674004 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:02.674014 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:02.674017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:02.674218 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:02.674622 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:02.674628 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:02.674631 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:02.675026 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:02.675391 (+ 365us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:02.675740 (+ 349us) server_negotiation.cc:299] Negotiation successful
0808 13:07:02.675972 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 13:07:02.677269 25140 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:07:02.813568 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 24963
I20260808 13:07:02.841442 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 24404
I20260808 13:07:02.842741 24496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:07:02.957247 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 24404
I20260808 13:07:02.989859 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:43305
--webserver_interface=127.1.124.190
--webserver_port=45395
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:35975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:43305
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:07:03.342535 25222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:07:03.342934 25222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:07:03.343104 25222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:07:03.350708 25222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:07:03.350828 25222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:07:03.350927 25222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:07:03.351034 25222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:07:03.351114 25222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:07:03.361634 25222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:35975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:43305
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:43305
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=45395
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:07:03.363029 25222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:07:03.364522 25222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:07:03.376739 25229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:03.376776 25230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:03.377454 25232 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:07:03.377813 25222 server_base.cc:1034] running on GCE node
I20260808 13:07:03.378641 25222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:07:03.380129 25222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:07:03.381386 25222 hybrid_clock.cc:648] HybridClock initialized: now 1786194423381323 us; error 48 us; skew 500 ppm
I20260808 13:07:03.381776 25222 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:07:03 dist-test-slave-t3q3 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194423, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:07:03.385439 25222 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:07:03.387012 25222 webserver.cc:460] Webserver started at http://127.1.124.190:45395/ using document root <none> and password file <none>
I20260808 13:07:03.387739 25222 fs_manager.cc:359] Metadata directory not provided
I20260808 13:07:03.387897 25222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:07:03.393715 25222 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 13:07:03.396709 25242 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:03.398006 25222 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:07:03.398231 25222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "791268c5f57843bba2907ebd3433e77a"
format_stamp: "Formatted at 2026-08-08 13:06:57 on dist-test-slave-t3q3"
server_key: "ee19eb0b9185b9b68ded49d0f001b21e"
server_key_iv: "6b3f6dc453ccf0e91da6e5259702a0ff"
server_key_version: "encryptionkey@0"
I20260808 13:07:03.399514 25222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:07:03.414553 25222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:07:03.418983 25222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:07:03.419276 25222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:07:03.438817 25222 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:43305
I20260808 13:07:03.438838 25311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:43305 every 8 connection(s)
I20260808 13:07:03.440428 25222 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:07:03.447343 25312 sys_catalog.cc:263] Verifying existing consensus state
I20260808 13:07:03.449098 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 25222
I20260808 13:07:03.450150 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:35215
--local_ip_for_outbound_sockets=127.1.124.129
--tserver_master_addrs=127.1.124.190:43305
--webserver_port=38565
--webserver_interface=127.1.124.129
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:35975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:07:03.451740 25312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a: Bootstrap starting.
I20260808 13:07:03.476744 25312 log.cc:824] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a: Log is configured to *not* fsync() on all Append() calls
I20260808 13:07:03.493251 25312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 13:07:03.493822 25312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a: Bootstrap complete.
I20260808 13:07:03.503388 25312 raft_consensus.cc:348] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791268c5f57843bba2907ebd3433e77a" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43305 } }
I20260808 13:07:03.504648 25312 raft_consensus.cc:729] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 791268c5f57843bba2907ebd3433e77a, State: Initialized, Role: FOLLOWER
I20260808 13:07:03.505286 25312 consensus_queue.cc:260] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791268c5f57843bba2907ebd3433e77a" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43305 } }
I20260808 13:07:03.505527 25312 raft_consensus.cc:388] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:07:03.505649 25312 raft_consensus.cc:482] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:07:03.505851 25312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 2 FOLLOWER]: Advancing to term 3
I20260808 13:07:03.509878 25312 raft_consensus.cc:504] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791268c5f57843bba2907ebd3433e77a" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43305 } }
I20260808 13:07:03.510443 25312 leader_election.cc:302] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 791268c5f57843bba2907ebd3433e77a; no voters:
I20260808 13:07:03.511307 25312 leader_election.cc:288] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 13:07:03.511457 25320 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 3 FOLLOWER]: Leader election won for term 3
I20260808 13:07:03.512657 25320 raft_consensus.cc:686] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [term 3 LEADER]: Becoming Leader. State: Replica: 791268c5f57843bba2907ebd3433e77a, State: Running, Role: LEADER
I20260808 13:07:03.513367 25320 consensus_queue.cc:237] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791268c5f57843bba2907ebd3433e77a" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43305 } }
I20260808 13:07:03.513826 25312 sys_catalog.cc:564] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:07:03.516278 25321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "791268c5f57843bba2907ebd3433e77a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791268c5f57843bba2907ebd3433e77a" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43305 } } }
I20260808 13:07:03.516745 25321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [sys.catalog]: This master's current role is: LEADER
I20260808 13:07:03.520607 25322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 791268c5f57843bba2907ebd3433e77a. Latest consensus state: current_term: 3 leader_uuid: "791268c5f57843bba2907ebd3433e77a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "791268c5f57843bba2907ebd3433e77a" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 43305 } } }
I20260808 13:07:03.520999 25322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 791268c5f57843bba2907ebd3433e77a [sys.catalog]: This master's current role is: LEADER
I20260808 13:07:03.521492 25326 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:07:03.532696 25326 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:07:03.534499 25326 catalog_manager.cc:1259] Loaded cluster ID: 758e54337868452d9506f08c2a124c67
I20260808 13:07:03.534706 25326 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 13:07:03.537530 25326 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe9285f69ad google::LogMessage::Fail() at ??:0
@ 0x7fe9285f988a google::LogMessage::SendToLog() at ??:0
@ 0x7fe9285f64de google::LogMessage::Flush() at ??:0
@ 0x7fe9285f7a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe93e029512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fe93e026976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe93e0ae610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fe93e0ae439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe92b242345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe92b24591c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fe92b245799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fe92b220eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe92e4fa6db start_thread at ??:0
@ 0x7fe92749061f clone at ??:0
W20260808 13:07:03.852849 25315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:07:03.853225 25315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:07:03.853384 25315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:07:03.861028 25315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:07:03.861202 25315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:07:03.861413 25315 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:07:03.871591 25315 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:35975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:35215
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=38565
--tserver_master_addrs=127.1.124.190:43305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:07:03.873065 25315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:07:03.874449 25315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:07:03.887984 25385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:03.888020 25383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:03.888024 25380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:07:03.889001 25315 server_base.cc:1034] running on GCE node
I20260808 13:07:03.889654 25315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:07:03.890626 25315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:07:03.891888 25315 hybrid_clock.cc:648] HybridClock initialized: now 1786194423891846 us; error 46 us; skew 500 ppm
I20260808 13:07:03.892320 25315 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:07:03 dist-test-slave-t3q3 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194423, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:07:03.900550 25315 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:07:03.902323 25315 webserver.cc:460] Webserver started at http://127.1.124.129:38565/ using document root <none> and password file <none>
I20260808 13:07:03.903141 25315 fs_manager.cc:359] Metadata directory not provided
I20260808 13:07:03.903308 25315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:07:03.909034 25315 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 13:07:03.911936 25394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:03.913548 25315 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 13:07:03.913774 25315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "4a485f59cf414ba0bb3fedb581b344dd"
format_stamp: "Formatted at 2026-08-08 13:06:58 on dist-test-slave-t3q3"
server_key: "45a6a534451f30fb91ee2eec3f4ba46f"
server_key_iv: "3f4e95820fcfb25c5dcfad2283cccb71"
server_key_version: "encryptionkey@0"
I20260808 13:07:03.914837 25315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:07:03.929359 25315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:07:03.934119 25315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:07:03.934437 25315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:07:03.935479 25315 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:07:03.937000 25315 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:07:03.937145 25315 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:03.937287 25315 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:07:03.937397 25315 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:03.974550 25315 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:35215
I20260808 13:07:03.974742 25560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:35215 every 8 connection(s)
I20260808 13:07:03.976154 25315 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:07:03.982184 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:03.979550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.129:56681)
0808 13:07:03.980755 (+ 1205us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:03.981141 (+ 386us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":866,"thread_start_us":606,"threads_started":1}
W20260808 13:07:03.982903 25561 heartbeater.cc:644] Failed to heartbeat to 127.1.124.190:43305 (0 consecutive failures): Network error: Failed to ping master at 127.1.124.190:43305: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
I20260808 13:07:03.983784 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:03.983438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.129:57163)
0808 13:07:03.983595 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:03.983677 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 13:07:03.984026 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 25315
I20260808 13:07:03.984732 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:37813
--local_ip_for_outbound_sockets=127.1.124.130
--tserver_master_addrs=127.1.124.190:43305
--webserver_port=36307
--webserver_interface=127.1.124.130
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:35975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:07:03.984869 25563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:03.984509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.129:52881)
0808 13:07:03.984649 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:03.984738 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 13:07:03.985105 25561 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 13:07:04.344642 25564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:07:04.344940 25564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:07:04.345122 25564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:07:04.352411 25564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:07:04.352528 25564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:07:04.352752 25564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:07:04.363366 25564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:35975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:37813
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=36307
--tserver_master_addrs=127.1.124.190:43305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:07:04.364789 25564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:07:04.366164 25564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:07:04.379690 25581 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:04.379745 25578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:04.379776 25577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:07:04.380511 25564 server_base.cc:1034] running on GCE node
I20260808 13:07:04.381151 25564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:07:04.382202 25564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:07:04.383469 25564 hybrid_clock.cc:648] HybridClock initialized: now 1786194424383395 us; error 67 us; skew 500 ppm
I20260808 13:07:04.383846 25564 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:07:04 dist-test-slave-t3q3 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194424, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:07:04.387349 25564 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:07:04.389012 25564 webserver.cc:460] Webserver started at http://127.1.124.130:36307/ using document root <none> and password file <none>
I20260808 13:07:04.389751 25564 fs_manager.cc:359] Metadata directory not provided
I20260808 13:07:04.389905 25564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:07:04.397055 25564 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:07:04.401312 25590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:04.402897 25564 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260808 13:07:04.403077 25564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "6acc4ef52963473d993cc66899bbe5cb"
format_stamp: "Formatted at 2026-08-08 13:06:58 on dist-test-slave-t3q3"
server_key: "ca6c479dc37a2c95cc652c35527a960d"
server_key_iv: "3f2df92286f2ebba14fb05ff034a34c3"
server_key_version: "encryptionkey@0"
I20260808 13:07:04.404243 25564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:07:04.417151 25564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:07:04.421908 25564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:07:04.422214 25564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:07:04.423244 25564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:07:04.424721 25564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:07:04.424834 25564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:04.424973 25564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:07:04.425066 25564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:04.461318 25564 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:37813
I20260808 13:07:04.461414 25753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:37813 every 8 connection(s)
I20260808 13:07:04.462937 25564 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:07:04.466125 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 25564
I20260808 13:07:04.466800 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:44529
--local_ip_for_outbound_sockets=127.1.124.131
--tserver_master_addrs=127.1.124.190:43305
--webserver_port=42503
--webserver_interface=127.1.124.131
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:35975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:07:04.468413 25758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:04.466110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.130:55357)
0808 13:07:04.467052 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:04.467357 (+ 305us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":368,"threads_started":1}
W20260808 13:07:04.469092 25754 heartbeater.cc:644] Failed to heartbeat to 127.1.124.190:43305 (0 consecutive failures): Network error: Failed to ping master at 127.1.124.190:43305: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
I20260808 13:07:04.470060 25758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:04.469717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.130:49671)
0808 13:07:04.469873 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:04.469955 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 13:07:04.471017 25758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:04.470690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.130:35187)
0808 13:07:04.470821 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:04.470895 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 13:07:04.471323 25754 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 13:07:04.821332 25759 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:07:04.821631 25759 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:07:04.821805 25759 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:07:04.829212 25759 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:07:04.829344 25759 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:07:04.829545 25759 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:07:04.839720 25759 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:35975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:44529
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=42503
--tserver_master_addrs=127.1.124.190:43305
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:07:04.841182 25759 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:07:04.842581 25759 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:07:04.857013 25765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:04.856972 25767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:04.856930 25764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:07:04.858161 25759 server_base.cc:1034] running on GCE node
I20260808 13:07:04.858868 25759 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:07:04.859856 25759 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:07:04.861101 25759 hybrid_clock.cc:648] HybridClock initialized: now 1786194424861026 us; error 70 us; skew 500 ppm
I20260808 13:07:04.861441 25759 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:07:04 dist-test-slave-t3q3 krb5kdc[23582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194424, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:07:04.864945 25759 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:07:04.866425 25759 webserver.cc:460] Webserver started at http://127.1.124.131:42503/ using document root <none> and password file <none>
I20260808 13:07:04.867197 25759 fs_manager.cc:359] Metadata directory not provided
I20260808 13:07:04.867304 25759 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:07:04.872701 25759 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 13:07:04.875365 25779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:04.876811 25759 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:07:04.876989 25759 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "df1ac1b971934f54a95716037b089d34"
format_stamp: "Formatted at 2026-08-08 13:06:59 on dist-test-slave-t3q3"
server_key: "28463d66a8ebdeb2df6d4f46434dc890"
server_key_iv: "bedac146c41773fde0c222688632ba23"
server_key_version: "encryptionkey@0"
I20260808 13:07:04.877938 25759 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:07:04.900544 25759 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:07:04.904599 25759 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:07:04.904826 25759 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:07:04.905754 25759 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:07:04.907058 25759 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:07:04.907166 25759 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:04.907240 25759 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:07:04.907295 25759 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:04.941484 25759 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:44529
I20260808 13:07:04.941578 25943 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:44529 every 8 connection(s)
I20260808 13:07:04.942999 25759 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:07:04.947769 25946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:04.945992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.131:37957)
0808 13:07:04.946708 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:04.946956 (+ 248us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":269,"threads_started":1}
I20260808 13:07:04.948582 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 25759
W20260808 13:07:04.948601 25944 heartbeater.cc:644] Failed to heartbeat to 127.1.124.190:43305 (0 consecutive failures): Network error: Failed to ping master at 127.1.124.190:43305: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
I20260808 13:07:04.949579 25946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:04.949143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.131:53333)
0808 13:07:04.949360 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:04.949448 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 13:07:04.950506 25946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:04.950104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.131:40639)
0808 13:07:04.950254 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:04.950338 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 13:07:04.950721 25944 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 13:07:04.986542 25949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:04.985663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.129:43459)
0808 13:07:04.986185 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:04.986301 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":256,"threads_started":1}
I20260808 13:07:05.472795 25950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:05.471902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.130:36203)
0808 13:07:05.472440 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:05.472550 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":310,"threads_started":1}
I20260808 13:07:05.952373 25954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:05.951361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.131:54279)
0808 13:07:05.951958 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:05.952118 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":307,"threads_started":1}
I20260808 13:07:05.992919 25955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:05.987605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.129:35573)
0808 13:07:05.992563 (+ 4958us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:05.992694 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4787,"thread_start_us":4646,"threads_started":1}
I20260808 13:07:06.474596 25956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:06.473778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.130:34637)
0808 13:07:06.474330 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:06.474410 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":317,"threads_started":1}
I20260808 13:07:06.954275 25957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:06.953346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.131:49833)
0808 13:07:06.953903 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:06.954016 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":235,"threads_started":1}
I20260808 13:07:06.994824 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:06.994133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.129:33725)
0808 13:07:06.994557 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:06.994637 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":226,"threads_started":1}
I20260808 13:07:07.476294 25959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:07.475475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.130:45591)
0808 13:07:07.475953 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:07.476036 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":255,"threads_started":1}
I20260808 13:07:07.955890 25960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:07.955204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.131:33201)
0808 13:07:07.955622 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:07.955703 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":226,"threads_started":1}
I20260808 13:07:07.996651 25961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:07.995853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.129:46325)
0808 13:07:07.996396 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:07.996479 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":324,"threads_started":1}
I20260808 13:07:08.477882 25962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:08.477177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.130:46381)
0808 13:07:08.477612 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:08.477689 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":229,"threads_started":1}
I20260808 13:07:08.957816 25963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:08.957006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.131:49939)
0808 13:07:08.957535 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:08.957617 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":297,"threads_started":1}
I20260808 13:07:08.998746 25964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:08.997725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.129:35075)
0808 13:07:08.998390 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:08.998521 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":326,"threads_started":1}
I20260808 13:07:09.479509 25965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:09.478760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.130:51235)
0808 13:07:09.479254 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:09.479340 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":252,"threads_started":1}
I20260808 13:07:09.959473 25966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:09.958734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.131:36469)
0808 13:07:09.959217 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:09.959300 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":248,"threads_started":1}
I20260808 13:07:10.001343 25967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:09.999832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.129:50833)
0808 13:07:10.000517 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:10.000670 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":369,"threads_started":1}
I20260808 13:07:10.481252 25968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:10.480516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.130:58521)
0808 13:07:10.480983 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:10.481070 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":248,"threads_started":1}
I20260808 13:07:10.961216 25969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:10.960470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.131:42241)
0808 13:07:10.960928 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:10.961022 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":237,"threads_started":1}
I20260808 13:07:11.003286 25970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:11.002407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.129:34565)
0808 13:07:11.002948 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:11.003041 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":233,"threads_started":1}
I20260808 13:07:11.482908 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:11.482081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.130:40749)
0808 13:07:11.482599 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:11.482693 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":263,"threads_started":1}
I20260808 13:07:11.963251 25974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:11.962301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.131:46273)
0808 13:07:11.962883 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:11.962968 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":283,"threads_started":1}
I20260808 13:07:12.005110 25975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:12.004362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.129:34835)
0808 13:07:12.004848 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:12.004937 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":240,"threads_started":1}
I20260808 13:07:12.484616 25978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:12.483823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.130:54701)
0808 13:07:12.484345 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:12.484431 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":276,"threads_started":1}
I20260808 13:07:12.964982 25979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:12.964269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.131:36317)
0808 13:07:12.964733 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:12.964817 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":252,"threads_started":1}
I20260808 13:07:13.007519 25980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:13.006275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.129:59315)
0808 13:07:13.007077 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:13.007226 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":388,"threads_started":1}
I20260808 13:07:13.486517 25981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:13.485673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.130:56923)
0808 13:07:13.486226 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:13.486314 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":291,"threads_started":1}
I20260808 13:07:13.966751 25982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:13.965930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.131:56725)
0808 13:07:13.966458 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:13.966569 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":272,"threads_started":1}
I20260808 13:07:14.009425 25985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:14.008571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.129:53553)
0808 13:07:14.009179 (+ 608us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:14.009261 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":340,"threads_started":1}
I20260808 13:07:14.488358 25986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:14.487549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.130:42355)
0808 13:07:14.488038 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:14.488189 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":280,"threads_started":1}
I20260808 13:07:14.968686 25987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:14.967811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.131:35467)
0808 13:07:14.968417 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:14.968514 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":391,"threads_started":1}
I20260808 13:07:15.011439 25988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:15.010500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.129:59525)
0808 13:07:15.011141 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:15.011249 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":247,"threads_started":1}
I20260808 13:07:15.490135 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:15.489348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.130:52761)
0808 13:07:15.489842 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:15.489939 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":261,"threads_started":1}
I20260808 13:07:15.970748 25990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:15.969793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.131:60105)
0808 13:07:15.970340 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:15.970475 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":305,"threads_started":1}
I20260808 13:07:16.013209 25992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:16.012473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.129:57625)
0808 13:07:16.012945 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:16.013044 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":248,"threads_started":1}
I20260808 13:07:16.491887 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:16.491153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.130:34603)
0808 13:07:16.491628 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:16.491711 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":237,"threads_started":1}
I20260808 13:07:16.972636 25994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:16.971825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.131:46707)
0808 13:07:16.972346 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:16.972445 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":307,"threads_started":1}
I20260808 13:07:17.015151 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:17.014278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.129:48161)
0808 13:07:17.014843 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:17.014952 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":286,"threads_started":1}
I20260808 13:07:17.493989 25997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:17.493075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.130:33227)
0808 13:07:17.493641 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:17.493794 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":305,"threads_started":1}
I20260808 13:07:17.974453 25999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:17.973670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.131:41259)
0808 13:07:17.974135 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:17.974266 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":245,"threads_started":1}
I20260808 13:07:18.017118 26000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:18.016245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.129:33239)
0808 13:07:18.016726 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:18.016850 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":222,"threads_started":1}
I20260808 13:07:18.495769 26001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:18.494948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.130:41037)
0808 13:07:18.495449 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:18.495540 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":240,"threads_started":1}
I20260808 13:07:18.976365 26002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:18.975438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.131:54861)
0808 13:07:18.976002 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:18.976153 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":320,"threads_started":1}
I20260808 13:07:19.019363 26003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:19.018428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.129:34755)
0808 13:07:19.018927 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:19.019081 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":276,"threads_started":1}
I20260808 13:07:19.497759 26004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:19.496991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.130:45361)
0808 13:07:19.497472 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:19.497568 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":246,"threads_started":1}
I20260808 13:07:19.956337 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 25315
I20260808 13:07:19.974491 26005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:19.957777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38488 (local address 127.1.124.129:35215)
0808 13:07:19.958352 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:19.958365 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:19.958405 (+ 40us) server_negotiation.cc:407] Connection header received
0808 13:07:19.959934 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:19.959944 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:19.960462 (+ 518us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:19.960756 (+ 294us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:19.962167 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:19.963467 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:19.964813 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:19.965487 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:19.965626 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:19.968291 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:19.968299 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:19.968315 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:19.968368 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:19.971376 (+ 3008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:19.972129 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:19.972136 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:19.972143 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:19.972311 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:19.972690 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:19.972694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:19.972697 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:19.973275 (+ 578us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:19.973509 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:19.973872 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 13:07:19.974217 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":274,"threads_started":1}
I20260808 13:07:19.976043 25555 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:07:19.978688 26010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:19.977643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:43305 (local address 127.1.124.131:43453)
0808 13:07:19.978235 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:19.978398 (+ 163us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:43305: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":284,"threads_started":1}
I20260808 13:07:20.064641 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 25315
I20260808 13:07:20.091566 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 25564
I20260808 13:07:20.110970 26013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:20.092697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59818 (local address 127.1.124.130:37813)
0808 13:07:20.093274 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:20.093289 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:20.093328 (+ 39us) server_negotiation.cc:407] Connection header received
0808 13:07:20.094989 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:20.095000 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:20.095444 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:20.095775 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:20.097465 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:20.098769 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:20.100237 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:20.100946 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:20.101098 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:20.104214 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:20.104224 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:20.104242 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:20.104282 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:20.107224 (+ 2942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:20.108136 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:20.108146 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:20.108156 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:20.108375 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:20.108791 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:20.108796 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:20.108801 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:20.109373 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:20.109759 (+ 386us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:20.110195 (+ 436us) server_negotiation.cc:299] Negotiation successful
0808 13:07:20.110658 (+ 463us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":264,"threads_started":1}
I20260808 13:07:20.112648 25743 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:07:20.199581 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 25564
I20260808 13:07:20.226460 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 25759
I20260808 13:07:20.244984 26015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:20.227479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51252 (local address 127.1.124.131:44529)
0808 13:07:20.228181 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:20.228196 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:20.228241 (+ 45us) server_negotiation.cc:407] Connection header received
0808 13:07:20.229864 (+ 1623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:20.229874 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:20.230308 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:20.230622 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:20.232157 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:20.233477 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:20.234855 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:20.235549 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:20.235683 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:20.238723 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:20.238732 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:20.238748 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:20.238788 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:20.241821 (+ 3033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:20.242521 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:20.242527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:20.242534 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:20.242715 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:20.243082 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:20.243085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:20.243087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:20.243623 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:20.243854 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:20.244330 (+ 476us) server_negotiation.cc:299] Negotiation successful
0808 13:07:20.244683 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":303,"threads_started":1}
I20260808 13:07:20.246431 25936 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:07:20.330502 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 25759
2026-08-08T13:07:20Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24986 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:07:20 dist-test-slave-t3q3 krb5kdc[26021](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:07:20 dist-test-slave-t3q3 krb5kdc[26021](info): set up 2 sockets
Aug 08 13:07:20 dist-test-slave-t3q3 krb5kdc[26021](info): commencing operation
krb5kdc: starting...
W20260808 13:07:21.675148 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.295s user 0.004s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:07:21 dist-test-slave-t3q3 krb5kdc[26021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:07:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:07:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:07:21.987569 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:39149
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:45281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:39149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:07:22.346694 26041 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:07:22.346990 26041 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:07:22.347191 26041 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:07:22.354585 26041 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 13:07:22.354699 26041 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 13:07:22.354784 26041 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 13:07:22.354866 26041 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:07:22.354961 26041 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:07:22.355082 26041 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:07:22.355168 26041 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:07:22.355254 26041 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:07:22.365218 26041 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:45281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:39149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:39149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:07:22.366595 26041 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:07:22.368018 26041 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:07:22.380671 26053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:22.380643 26056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:22.380746 26052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:07:22.381136 26041 server_base.cc:1034] running on GCE node
I20260808 13:07:22.382107 26041 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:07:22.383476 26041 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:07:22.384709 26041 hybrid_clock.cc:648] HybridClock initialized: now 1786194442384644 us; error 31 us; skew 500 ppm
I20260808 13:07:22.385067 26041 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:07:22 dist-test-slave-t3q3 krb5kdc[26021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194442, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:07:22.392935 26041 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:07:22.394582 26041 webserver.cc:460] Webserver started at http://127.1.124.190:43803/ using document root <none> and password file <none>
I20260808 13:07:22.395313 26041 fs_manager.cc:359] Metadata directory not provided
I20260808 13:07:22.395452 26041 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:07:22.395891 26041 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:07:22.399111 26041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "da2f58b07de549af9773d27e893bdb90"
format_stamp: "Formatted at 2026-08-08 13:07:22 on dist-test-slave-t3q3"
server_key: "bb983c92fb988c4423aef9a11fd102af"
server_key_iv: "47173ca5213611f84bf756385a9d7a55"
server_key_version: "encryptionkey@0"
I20260808 13:07:22.399948 26041 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "da2f58b07de549af9773d27e893bdb90"
format_stamp: "Formatted at 2026-08-08 13:07:22 on dist-test-slave-t3q3"
server_key: "bb983c92fb988c4423aef9a11fd102af"
server_key_iv: "47173ca5213611f84bf756385a9d7a55"
server_key_version: "encryptionkey@0"
I20260808 13:07:22.405303 26041 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 13:07:22.409107 26067 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:22.410538 26041 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:07:22.410749 26041 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "da2f58b07de549af9773d27e893bdb90"
format_stamp: "Formatted at 2026-08-08 13:07:22 on dist-test-slave-t3q3"
server_key: "bb983c92fb988c4423aef9a11fd102af"
server_key_iv: "47173ca5213611f84bf756385a9d7a55"
server_key_version: "encryptionkey@0"
I20260808 13:07:22.410995 26041 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:07:22.427459 26041 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:07:22.431828 26041 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:07:22.432183 26041 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:07:22.451957 26041 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:39149
I20260808 13:07:22.451979 26143 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:39149 every 8 connection(s)
I20260808 13:07:22.453577 26041 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:07:22.457645 26145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:07:22.459859 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 26041
I20260808 13:07:22.460227 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:07:22.460947 1522 external_mini_cluster.cc:1428] Setting key 91b216b8d1b2a66e0984d38b35fb2885
I20260808 13:07:22.469563 26145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90: Bootstrap starting.
I20260808 13:07:22.472882 26145 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90: Neither blocks nor log segments found. Creating new log.
I20260808 13:07:22.473968 26145 log.cc:824] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90: Log is configured to *not* fsync() on all Append() calls
Aug 08 13:07:22 dist-test-slave-t3q3 krb5kdc[26021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:07:22.477209 26145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90: No bootstrap required, opened a new log
I20260808 13:07:22.484844 26150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:22.463111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45568 (local address 127.1.124.190:39149)
0808 13:07:22.464001 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:22.464013 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:22.464120 (+ 107us) server_negotiation.cc:407] Connection header received
0808 13:07:22.465895 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:22.465923 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:22.466318 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:22.466753 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:22.468379 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:22.469756 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:22.471163 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:22.471853 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:22.472050 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:22.476977 (+ 4927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:22.476985 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:22.477001 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:22.477036 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:22.480541 (+ 3505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:22.481297 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:22.481305 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:22.481311 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:22.481488 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:22.481911 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:22.481917 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:22.481922 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:22.482487 (+ 565us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:22.482843 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:22.483273 (+ 430us) server_negotiation.cc:299] Negotiation successful
0808 13:07:22.483733 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":419,"threads_started":1}
I20260808 13:07:22.485652 26145 raft_consensus.cc:348] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2f58b07de549af9773d27e893bdb90" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39149 } }
I20260808 13:07:22.486030 26145 raft_consensus.cc:374] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:07:22.486157 26145 raft_consensus.cc:729] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da2f58b07de549af9773d27e893bdb90, State: Initialized, Role: FOLLOWER
I20260808 13:07:22.486775 26145 consensus_queue.cc:260] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2f58b07de549af9773d27e893bdb90" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39149 } }
I20260808 13:07:22.487043 26145 raft_consensus.cc:388] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:07:22.487200 26145 raft_consensus.cc:482] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:07:22.487386 26145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:07:22.489888 26145 raft_consensus.cc:504] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2f58b07de549af9773d27e893bdb90" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39149 } }
I20260808 13:07:22.490387 26145 leader_election.cc:302] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da2f58b07de549af9773d27e893bdb90; no voters:
I20260808 13:07:22.491001 26145 leader_election.cc:288] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:07:22.491160 26154 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:07:22.491937 26154 raft_consensus.cc:686] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 1 LEADER]: Becoming Leader. State: Replica: da2f58b07de549af9773d27e893bdb90, State: Running, Role: LEADER
I20260808 13:07:22.492630 26154 consensus_queue.cc:237] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2f58b07de549af9773d27e893bdb90" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39149 } }
I20260808 13:07:22.493245 26145 sys_catalog.cc:564] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:07:22.495605 26155 sys_catalog.cc:455] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da2f58b07de549af9773d27e893bdb90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2f58b07de549af9773d27e893bdb90" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39149 } } }
I20260808 13:07:22.495630 26157 sys_catalog.cc:455] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da2f58b07de549af9773d27e893bdb90. Latest consensus state: current_term: 1 leader_uuid: "da2f58b07de549af9773d27e893bdb90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2f58b07de549af9773d27e893bdb90" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39149 } } }
I20260808 13:07:22.496230 26155 sys_catalog.cc:458] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [sys.catalog]: This master's current role is: LEADER
I20260808 13:07:22.496230 26157 sys_catalog.cc:458] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [sys.catalog]: This master's current role is: LEADER
I20260808 13:07:22.498286 26162 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:07:22.505018 26162 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:07:22.515704 26162 catalog_manager.cc:1347] Generated new cluster ID: 3d8168421f97430abc3d8d1aa248360a
I20260808 13:07:22.515849 26162 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:07:22.527218 26162 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:07:22.528625 26162 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:07:22.541697 26162 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90: Generated new TSK 0
I20260808 13:07:22.542711 26162 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.124.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
I20260808 13:07:22.628516 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:39149
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:45281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:07:22.990631 26192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:07:22.990988 26192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:07:22.991159 26192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:07:22.998979 26192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:07:22.999147 26192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:07:22.999338 26192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:07:23.009414 26192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:45281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:39149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:07:23.010824 26192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:07:23.012442 26192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:07:23.026578 26199 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:23.026633 26206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:23.026605 26202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:07:23.027470 26192 server_base.cc:1034] running on GCE node
I20260808 13:07:23.028126 26192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:07:23.029134 26192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:07:23.030344 26192 hybrid_clock.cc:648] HybridClock initialized: now 1786194443030288 us; error 61 us; skew 500 ppm
I20260808 13:07:23.030622 26192 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:07:23 dist-test-slave-t3q3 krb5kdc[26021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194443, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:07:23.033941 26192 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:07:23.035394 26192 webserver.cc:460] Webserver started at http://127.1.124.129:46633/ using document root <none> and password file <none>
I20260808 13:07:23.036039 26192 fs_manager.cc:359] Metadata directory not provided
I20260808 13:07:23.036227 26192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:07:23.036518 26192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:07:23.039479 26192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "766511b6ae0544c3a8eabf2ed87c344b"
format_stamp: "Formatted at 2026-08-08 13:07:23 on dist-test-slave-t3q3"
server_key: "47193f2a5ab60fb8c51e9a26256d237b"
server_key_iv: "eac434ea418f52172c97240e134831c3"
server_key_version: "encryptionkey@0"
I20260808 13:07:23.040289 26192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "766511b6ae0544c3a8eabf2ed87c344b"
format_stamp: "Formatted at 2026-08-08 13:07:23 on dist-test-slave-t3q3"
server_key: "47193f2a5ab60fb8c51e9a26256d237b"
server_key_iv: "eac434ea418f52172c97240e134831c3"
server_key_version: "encryptionkey@0"
I20260808 13:07:23.045595 26192 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:07:23.049666 26214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:23.052275 26192 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.001s
I20260808 13:07:23.052467 26192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "766511b6ae0544c3a8eabf2ed87c344b"
format_stamp: "Formatted at 2026-08-08 13:07:23 on dist-test-slave-t3q3"
server_key: "47193f2a5ab60fb8c51e9a26256d237b"
server_key_iv: "eac434ea418f52172c97240e134831c3"
server_key_version: "encryptionkey@0"
I20260808 13:07:23.052695 26192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:07:23.096426 26192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:07:23.101351 26192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:07:23.101706 26192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:07:23.102772 26192 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:07:23.104350 26192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:07:23.104458 26192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:23.104616 26192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:07:23.104699 26192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:23.139628 26192 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:42343
I20260808 13:07:23.139694 26376 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:42343 every 8 connection(s)
I20260808 13:07:23.140978 26192 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:07:23.143239 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 26192
I20260808 13:07:23.143532 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:07:23.144495 1522 external_mini_cluster.cc:1428] Setting key 6d331500709c2592ef34b00c0f470951
Aug 08 13:07:23 dist-test-slave-t3q3 krb5kdc[26021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194443, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:07:23.166216 26380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:23.144155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:43831 (local address 127.1.124.190:39149)
0808 13:07:23.144675 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:23.144684 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:23.145791 (+ 1107us) server_negotiation.cc:407] Connection header received
0808 13:07:23.148372 (+ 2581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:23.148376 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:23.148442 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:23.148530 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:23.150815 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:23.151580 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:23.154395 (+ 2815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:23.154888 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:23.155045 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:23.159295 (+ 4250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:23.159301 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:23.159304 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:23.159339 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:23.162586 (+ 3247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:23.163413 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:23.163418 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:23.163421 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:23.163553 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:23.164101 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:23.164106 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:23.164109 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:23.164399 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:23.164581 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:23.165580 (+ 999us) server_negotiation.cc:299] Negotiation successful
0808 13:07:23.165830 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":224,"threads_started":1}
I20260808 13:07:23.167291 26382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:23.144791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.129:43831)
0808 13:07:23.145613 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:23.145663 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:23.148019 (+ 2356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:23.148831 (+ 812us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:23.148848 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:23.149705 (+ 857us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:07:23.150557 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:23.150587 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:23.151765 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:23.151773 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:23.154212 (+ 2439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:23.154232 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:23.155159 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:23.155169 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:23.155517 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:23.156457 (+ 940us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:07:23.156488 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:07:23.159043 (+ 2555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:07:23.162756 (+ 3713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:07:23.162766 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:07:23.162784 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:07:23.163262 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:07:23.163675 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:07:23.163680 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:07:23.163684 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:07:23.163946 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:07:23.164550 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:07:23.164559 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:07:23.165160 (+ 601us) client_negotiation.cc:769] Sending connection context
0808 13:07:23.165452 (+ 292us) client_negotiation.cc:240] Negotiation successful
0808 13:07:23.166029 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":323,"threads_started":1}
I20260808 13:07:23.169319 26377 heartbeater.cc:347] Connected to a master server at 127.1.124.190:39149
I20260808 13:07:23.169662 26377 heartbeater.cc:464] Registering TS with master...
I20260808 13:07:23.170449 26377 heartbeater.cc:511] Master 127.1.124.190:39149 requested a full tablet report, sending...
I20260808 13:07:23.172818 26090 ts_manager.cc:194] Registered new tserver with Master: 766511b6ae0544c3a8eabf2ed87c344b (127.1.124.129:42343)
I20260808 13:07:23.174405 26090 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:43831
WARNING: no policy specified for kudu/127.1.124.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
I20260808 13:07:23.215807 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:39149
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:45281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:07:23.560174 26389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:07:23.560496 26389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:07:23.560667 26389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:07:23.567857 26389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:07:23.568022 26389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:07:23.568233 26389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:07:23.578387 26389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:45281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:39149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:07:23.579763 26389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:07:23.581198 26389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:07:23.595016 26395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:23.594996 26394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:23.595547 26397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:07:23.595902 26389 server_base.cc:1034] running on GCE node
I20260808 13:07:23.596549 26389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:07:23.597574 26389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:07:23.598774 26389 hybrid_clock.cc:648] HybridClock initialized: now 1786194443598676 us; error 92 us; skew 500 ppm
I20260808 13:07:23.599110 26389 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:07:23 dist-test-slave-t3q3 krb5kdc[26021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194443, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:07:23.602407 26389 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:07:23.604130 26389 webserver.cc:460] Webserver started at http://127.1.124.130:34421/ using document root <none> and password file <none>
I20260808 13:07:23.604857 26389 fs_manager.cc:359] Metadata directory not provided
I20260808 13:07:23.605016 26389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:07:23.605345 26389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:07:23.608443 26389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "0d0ea86bb8464914b0cd04ed16aba960"
format_stamp: "Formatted at 2026-08-08 13:07:23 on dist-test-slave-t3q3"
server_key: "ad2b3387acab3f88f20938211d1f38b1"
server_key_iv: "debb18a0e740dc23c6f066a4d24f60b8"
server_key_version: "encryptionkey@0"
I20260808 13:07:23.609223 26389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "0d0ea86bb8464914b0cd04ed16aba960"
format_stamp: "Formatted at 2026-08-08 13:07:23 on dist-test-slave-t3q3"
server_key: "ad2b3387acab3f88f20938211d1f38b1"
server_key_iv: "debb18a0e740dc23c6f066a4d24f60b8"
server_key_version: "encryptionkey@0"
I20260808 13:07:23.614444 26389 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 13:07:23.618148 26408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:23.619495 26389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:07:23.619695 26389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "0d0ea86bb8464914b0cd04ed16aba960"
format_stamp: "Formatted at 2026-08-08 13:07:23 on dist-test-slave-t3q3"
server_key: "ad2b3387acab3f88f20938211d1f38b1"
server_key_iv: "debb18a0e740dc23c6f066a4d24f60b8"
server_key_version: "encryptionkey@0"
I20260808 13:07:23.619910 26389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:07:23.635998 26389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:07:23.640800 26389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:07:23.641121 26389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:07:23.642207 26389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:07:23.643741 26389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:07:23.643852 26389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:23.644022 26389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:07:23.644137 26389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260808 13:07:23.679692 26389 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:46503
I20260808 13:07:23.679734 26578 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:46503 every 8 connection(s)
I20260808 13:07:23.681107 26389 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:07:23.687794 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 26389
I20260808 13:07:23.688030 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:07:23.688666 1522 external_mini_cluster.cc:1428] Setting key 870119ad868115a2d823120b3735129b
Aug 08 13:07:23 dist-test-slave-t3q3 krb5kdc[26021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194443, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:07:23.704906 26584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:23.683798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:41303 (local address 127.1.124.190:39149)
0808 13:07:23.684339 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:23.684346 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:23.685325 (+ 979us) server_negotiation.cc:407] Connection header received
0808 13:07:23.687093 (+ 1768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:23.687097 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:23.687164 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:23.687272 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:23.689436 (+ 2164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:23.690314 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:23.693096 (+ 2782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:23.693625 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:23.693790 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:23.698568 (+ 4778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:23.698574 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:23.698577 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:23.698619 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:23.701537 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:23.702368 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:23.702373 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:23.702375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:23.702509 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:23.702988 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:23.702992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:23.702994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:23.703287 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:23.703467 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:23.704275 (+ 808us) server_negotiation.cc:299] Negotiation successful
0808 13:07:23.704514 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":314,"threads_started":1}
I20260808 13:07:23.705904 26585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:23.684434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.130:41303)
0808 13:07:23.685146 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:23.685192 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:23.686832 (+ 1640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:23.687593 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:23.687608 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:23.688406 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:07:23.689222 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:23.689249 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:23.690552 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:23.690559 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:23.692893 (+ 2334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:23.692941 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:23.693924 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:23.693936 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:23.694314 (+ 378us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:23.695099 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:07:23.695138 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:07:23.698262 (+ 3124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:07:23.701714 (+ 3452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:07:23.701725 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:07:23.701744 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:07:23.702220 (+ 476us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:07:23.702631 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:07:23.702636 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:07:23.702640 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:07:23.702850 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:07:23.703426 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:07:23.703436 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:07:23.703845 (+ 409us) client_negotiation.cc:769] Sending connection context
0808 13:07:23.704177 (+ 332us) client_negotiation.cc:240] Negotiation successful
0808 13:07:23.704746 (+ 569us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":279,"threads_started":1}
I20260808 13:07:23.707435 26581 heartbeater.cc:347] Connected to a master server at 127.1.124.190:39149
I20260808 13:07:23.707772 26581 heartbeater.cc:464] Registering TS with master...
I20260808 13:07:23.708634 26581 heartbeater.cc:511] Master 127.1.124.190:39149 requested a full tablet report, sending...
I20260808 13:07:23.710278 26090 ts_manager.cc:194] Registered new tserver with Master: 0d0ea86bb8464914b0cd04ed16aba960 (127.1.124.130:46503)
I20260808 13:07:23.711113 26090 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:41303
WARNING: no policy specified for kudu/127.1.124.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
I20260808 13:07:23.759548 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:39149
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:45281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:07:24.114691 26588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:07:24.115056 26588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:07:24.115163 26588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:07:24.122681 26588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:07:24.122797 26588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:07:24.122908 26588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:07:24.132788 26588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:45281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:39149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:07:24.134019 26588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:07:24.135298 26588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:07:24.148772 26595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:24.148747 26597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:24.148713 26601 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:07:24.149607 26588 server_base.cc:1034] running on GCE node
I20260808 13:07:24.150223 26588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:07:24.151172 26588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:07:24.152410 26588 hybrid_clock.cc:648] HybridClock initialized: now 1786194444152357 us; error 63 us; skew 500 ppm
I20260808 13:07:24.152753 26588 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:07:24 dist-test-slave-t3q3 krb5kdc[26021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194444, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:07:24.155946 26588 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:07:24.157588 26588 webserver.cc:460] Webserver started at http://127.1.124.131:39963/ using document root <none> and password file <none>
I20260808 13:07:24.158217 26588 fs_manager.cc:359] Metadata directory not provided
I20260808 13:07:24.158319 26588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:07:24.158586 26588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:07:24.161444 26588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "9db875a520d44bb693c809b4a6ea9537"
format_stamp: "Formatted at 2026-08-08 13:07:24 on dist-test-slave-t3q3"
server_key: "11977007e82c74b72c46429bb4df8250"
server_key_iv: "25a660ee84e4f7154333d89d115916d3"
server_key_version: "encryptionkey@0"
I20260808 13:07:24.162160 26588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "9db875a520d44bb693c809b4a6ea9537"
format_stamp: "Formatted at 2026-08-08 13:07:24 on dist-test-slave-t3q3"
server_key: "11977007e82c74b72c46429bb4df8250"
server_key_iv: "25a660ee84e4f7154333d89d115916d3"
server_key_version: "encryptionkey@0"
I20260808 13:07:24.167148 26588 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260808 13:07:24.170521 26614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:24.171866 26588 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:07:24.172045 26588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "9db875a520d44bb693c809b4a6ea9537"
format_stamp: "Formatted at 2026-08-08 13:07:24 on dist-test-slave-t3q3"
server_key: "11977007e82c74b72c46429bb4df8250"
server_key_iv: "25a660ee84e4f7154333d89d115916d3"
server_key_version: "encryptionkey@0"
I20260808 13:07:24.172299 26588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:07:24.177793 26377 heartbeater.cc:503] Master 127.1.124.190:39149 was elected leader, sending a full tablet report...
I20260808 13:07:24.186671 26588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:07:24.191184 26588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:07:24.191430 26588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:07:24.192452 26588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:07:24.193849 26588 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:07:24.193928 26588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:24.194044 26588 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:07:24.194114 26588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:24.229235 26588 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:34113
I20260808 13:07:24.229283 26782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:34113 every 8 connection(s)
I20260808 13:07:24.230584 26588 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:07:24.240984 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 26588
I20260808 13:07:24.241196 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:07:24.241762 1522 external_mini_cluster.cc:1428] Setting key 3bbd5a2dc2065e9d066c68b19ef5a87a
Aug 08 13:07:24 dist-test-slave-t3q3 krb5kdc[26021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194444, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:07:24.251991 26788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:24.233666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:59547 (local address 127.1.124.190:39149)
0808 13:07:24.234142 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:24.234149 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:24.235221 (+ 1072us) server_negotiation.cc:407] Connection header received
0808 13:07:24.237128 (+ 1907us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:24.237133 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:24.237204 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:24.237304 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:24.238831 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.239622 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.241787 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.242259 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.242399 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:24.246002 (+ 3603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:24.246007 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:24.246011 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:24.246043 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:24.248895 (+ 2852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:24.249646 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:24.249651 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:24.249653 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:24.249769 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:24.250219 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:24.250225 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:24.250233 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:24.250496 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:24.250676 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:24.251380 (+ 704us) server_negotiation.cc:299] Negotiation successful
0808 13:07:24.251611 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":238,"threads_started":1}
I20260808 13:07:24.252996 26791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:24.234292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.131:59547)
0808 13:07:24.235038 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:24.235080 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:24.236857 (+ 1777us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:24.237538 (+ 681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:24.237546 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:24.238087 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:07:24.238674 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:24.238693 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.239774 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.239779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:24.241636 (+ 1857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:24.241652 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.242512 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.242519 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:24.242759 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:24.243485 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:07:24.243509 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:07:24.245780 (+ 2271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:07:24.249048 (+ 3268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:07:24.249058 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:07:24.249075 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:07:24.249501 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:07:24.249883 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:07:24.249888 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:07:24.249892 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:07:24.250088 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:07:24.250629 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:07:24.250638 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:07:24.250992 (+ 354us) client_negotiation.cc:769] Sending connection context
0808 13:07:24.251275 (+ 283us) client_negotiation.cc:240] Negotiation successful
0808 13:07:24.251824 (+ 549us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":288,"threads_started":1}
I20260808 13:07:24.254354 26786 heartbeater.cc:347] Connected to a master server at 127.1.124.190:39149
I20260808 13:07:24.254702 26786 heartbeater.cc:464] Registering TS with master...
I20260808 13:07:24.255605 26786 heartbeater.cc:511] Master 127.1.124.190:39149 requested a full tablet report, sending...
I20260808 13:07:24.257632 26089 ts_manager.cc:194] Registered new tserver with Master: 9db875a520d44bb693c809b4a6ea9537 (127.1.124.131:34113)
I20260808 13:07:24.258567 26089 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:59547
I20260808 13:07:24.270676 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:07:24.289907 26788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:24.275871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45578 (local address 127.1.124.190:39149)
0808 13:07:24.276321 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:24.276330 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:24.276562 (+ 232us) server_negotiation.cc:407] Connection header received
0808 13:07:24.276774 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:24.276778 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:24.276846 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:24.276980 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:24.278530 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.279289 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.280863 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.281342 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.281454 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:24.284032 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:24.284038 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:24.284042 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:24.284146 (+ 104us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:24.286909 (+ 2763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:24.287614 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:24.287618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:24.287621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:24.287742 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:24.288483 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:24.288487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:24.288490 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:24.288823 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:24.289052 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:24.289457 (+ 405us) server_negotiation.cc:299] Negotiation successful
0808 13:07:24.289653 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 13:07:24.294791 26089 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45578:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:07:24.302310 26089 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:07:24.335037 26811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:24.324641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:42343 (local address 127.0.0.1:48752)
0808 13:07:24.325754 (+ 1113us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:24.325770 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:24.325878 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:24.327464 (+ 1586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:24.327468 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:24.327502 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:07:24.327959 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:24.327972 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.329933 (+ 1961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.329937 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:24.331882 (+ 1945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:24.331897 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.333899 (+ 2002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.333907 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:24.334122 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:24.334194 (+ 72us) client_negotiation.cc:769] Sending connection context
0808 13:07:24.334362 (+ 168us) client_negotiation.cc:240] Negotiation successful
0808 13:07:24.334593 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":986,"mutex_wait_us":174,"thread_start_us":829,"threads_started":1}
I20260808 13:07:24.335134 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:24.324939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48752 (local address 127.1.124.129:42343)
0808 13:07:24.326829 (+ 1890us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:24.326837 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:24.326863 (+ 26us) server_negotiation.cc:407] Connection header received
0808 13:07:24.326972 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:24.327012 (+ 40us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:24.327241 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:24.327460 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:07:24.328170 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.329749 (+ 1579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.332056 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.333677 (+ 1621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.333819 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:24.334065 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:24.334569 (+ 504us) server_negotiation.cc:299] Negotiation successful
0808 13:07:24.334762 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1715,"thread_start_us":1539,"threads_started":1}
I20260808 13:07:24.335314 26810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:24.317169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57502 (local address 127.1.124.131:34113)
0808 13:07:24.318149 (+ 980us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:24.318163 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:24.325565 (+ 7402us) server_negotiation.cc:407] Connection header received
0808 13:07:24.326074 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:24.326085 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:24.326377 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:24.326552 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:07:24.327760 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.329236 (+ 1476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.332255 (+ 3019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.333685 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.333819 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:24.334065 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:24.334774 (+ 709us) server_negotiation.cc:299] Negotiation successful
0808 13:07:24.334951 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":748,"thread_start_us":533,"threads_started":1}
I20260808 13:07:24.335424 26809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:24.316649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59584 (local address 127.1.124.130:46503)
0808 13:07:24.324855 (+ 8206us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:24.324870 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:24.326121 (+ 1251us) server_negotiation.cc:407] Connection header received
0808 13:07:24.326346 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:24.326353 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:24.326570 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:24.326757 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:07:24.328079 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.329701 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.332445 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.334016 (+ 1571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.334161 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:24.334343 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:24.334821 (+ 478us) server_negotiation.cc:299] Negotiation successful
0808 13:07:24.335006 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7985,"thread_start_us":9025,"threads_started":1}
I20260808 13:07:24.335973 26807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:24.316449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:34113 (local address 127.0.0.1:57502)
0808 13:07:24.325371 (+ 8922us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:24.325424 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:24.325754 (+ 330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:24.326869 (+ 1115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:24.326878 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:24.326910 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:07:24.327565 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:24.327581 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.329388 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.329395 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:24.332051 (+ 2656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:24.332092 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.334253 (+ 2161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.334261 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:24.334513 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:24.334589 (+ 76us) client_negotiation.cc:769] Sending connection context
0808 13:07:24.335373 (+ 784us) client_negotiation.cc:240] Negotiation successful
0808 13:07:24.335531 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8788,"mutex_wait_us":155,"thread_start_us":8654,"threads_started":1}
I20260808 13:07:24.337205 26804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:24.316235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:46503 (local address 127.0.0.1:59584)
0808 13:07:24.324034 (+ 7799us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:24.325970 (+ 1936us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:24.326158 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:24.326896 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:24.326900 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:24.326926 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:07:24.327783 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:24.327803 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.329871 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.329878 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:24.332297 (+ 2419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:24.332308 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.334270 (+ 1962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.334275 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:24.334585 (+ 310us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:24.334617 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 13:07:24.335818 (+ 1201us) client_negotiation.cc:240] Negotiation successful
0808 13:07:24.335926 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7494,"mutex_wait_us":80,"thread_start_us":7326,"threads_started":1}
I20260808 13:07:24.339514 26282 tablet_service.cc:1467] Processing CreateTablet for tablet 31ed8480f8934264beb7733e3c767bea (DEFAULT_TABLE table=test-table [id=baee8da3bd32425b8190b217a3e5bd00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:07:24.339823 26683 tablet_service.cc:1467] Processing CreateTablet for tablet 31ed8480f8934264beb7733e3c767bea (DEFAULT_TABLE table=test-table [id=baee8da3bd32425b8190b217a3e5bd00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:07:24.340948 26282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31ed8480f8934264beb7733e3c767bea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:07:24.340780 26481 tablet_service.cc:1467] Processing CreateTablet for tablet 31ed8480f8934264beb7733e3c767bea (DEFAULT_TABLE table=test-table [id=baee8da3bd32425b8190b217a3e5bd00]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:07:24.341382 26683 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31ed8480f8934264beb7733e3c767bea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:07:24.342150 26481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 31ed8480f8934264beb7733e3c767bea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:07:24.352823 26813 tablet_bootstrap.cc:492] T 31ed8480f8934264beb7733e3c767bea P 766511b6ae0544c3a8eabf2ed87c344b: Bootstrap starting.
I20260808 13:07:24.357322 26818 tablet_bootstrap.cc:492] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537: Bootstrap starting.
I20260808 13:07:24.357366 26813 tablet_bootstrap.cc:654] T 31ed8480f8934264beb7733e3c767bea P 766511b6ae0544c3a8eabf2ed87c344b: Neither blocks nor log segments found. Creating new log.
I20260808 13:07:24.358368 26815 tablet_bootstrap.cc:492] T 31ed8480f8934264beb7733e3c767bea P 0d0ea86bb8464914b0cd04ed16aba960: Bootstrap starting.
I20260808 13:07:24.358942 26813 log.cc:824] T 31ed8480f8934264beb7733e3c767bea P 766511b6ae0544c3a8eabf2ed87c344b: Log is configured to *not* fsync() on all Append() calls
I20260808 13:07:24.362154 26815 tablet_bootstrap.cc:654] T 31ed8480f8934264beb7733e3c767bea P 0d0ea86bb8464914b0cd04ed16aba960: Neither blocks nor log segments found. Creating new log.
I20260808 13:07:24.363013 26818 tablet_bootstrap.cc:654] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537: Neither blocks nor log segments found. Creating new log.
I20260808 13:07:24.363543 26815 log.cc:824] T 31ed8480f8934264beb7733e3c767bea P 0d0ea86bb8464914b0cd04ed16aba960: Log is configured to *not* fsync() on all Append() calls
I20260808 13:07:24.364138 26813 tablet_bootstrap.cc:492] T 31ed8480f8934264beb7733e3c767bea P 766511b6ae0544c3a8eabf2ed87c344b: No bootstrap required, opened a new log
I20260808 13:07:24.364326 26818 log.cc:824] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537: Log is configured to *not* fsync() on all Append() calls
I20260808 13:07:24.364530 26813 ts_tablet_manager.cc:1397] T 31ed8480f8934264beb7733e3c767bea P 766511b6ae0544c3a8eabf2ed87c344b: Time spent bootstrapping tablet: real 0.012s user 0.008s sys 0.000s
I20260808 13:07:24.366817 26815 tablet_bootstrap.cc:492] T 31ed8480f8934264beb7733e3c767bea P 0d0ea86bb8464914b0cd04ed16aba960: No bootstrap required, opened a new log
I20260808 13:07:24.367110 26815 ts_tablet_manager.cc:1397] T 31ed8480f8934264beb7733e3c767bea P 0d0ea86bb8464914b0cd04ed16aba960: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260808 13:07:24.367558 26818 tablet_bootstrap.cc:492] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537: No bootstrap required, opened a new log
I20260808 13:07:24.367748 26818 ts_tablet_manager.cc:1397] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.003s
I20260808 13:07:24.375349 26818 raft_consensus.cc:348] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:07:24.375541 26813 raft_consensus.cc:348] T 31ed8480f8934264beb7733e3c767bea P 766511b6ae0544c3a8eabf2ed87c344b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:07:24.375883 26818 raft_consensus.cc:374] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:07:24.376024 26818 raft_consensus.cc:729] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9db875a520d44bb693c809b4a6ea9537, State: Initialized, Role: FOLLOWER
I20260808 13:07:24.376024 26813 raft_consensus.cc:374] T 31ed8480f8934264beb7733e3c767bea P 766511b6ae0544c3a8eabf2ed87c344b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:07:24.376294 26813 raft_consensus.cc:729] T 31ed8480f8934264beb7733e3c767bea P 766511b6ae0544c3a8eabf2ed87c344b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 766511b6ae0544c3a8eabf2ed87c344b, State: Initialized, Role: FOLLOWER
I20260808 13:07:24.377008 26818 consensus_queue.cc:260] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:07:24.376972 26813 consensus_queue.cc:260] T 31ed8480f8934264beb7733e3c767bea P 766511b6ae0544c3a8eabf2ed87c344b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:07:24.377919 26815 raft_consensus.cc:348] T 31ed8480f8934264beb7733e3c767bea P 0d0ea86bb8464914b0cd04ed16aba960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:07:24.378428 26815 raft_consensus.cc:374] T 31ed8480f8934264beb7733e3c767bea P 0d0ea86bb8464914b0cd04ed16aba960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:07:24.378578 26815 raft_consensus.cc:729] T 31ed8480f8934264beb7733e3c767bea P 0d0ea86bb8464914b0cd04ed16aba960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d0ea86bb8464914b0cd04ed16aba960, State: Initialized, Role: FOLLOWER
I20260808 13:07:24.379037 26813 ts_tablet_manager.cc:1428] T 31ed8480f8934264beb7733e3c767bea P 766511b6ae0544c3a8eabf2ed87c344b: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260808 13:07:24.379390 26815 consensus_queue.cc:260] T 31ed8480f8934264beb7733e3c767bea P 0d0ea86bb8464914b0cd04ed16aba960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:07:24.379787 26786 heartbeater.cc:503] Master 127.1.124.190:39149 was elected leader, sending a full tablet report...
I20260808 13:07:24.381266 26581 heartbeater.cc:503] Master 127.1.124.190:39149 was elected leader, sending a full tablet report...
I20260808 13:07:24.382041 26815 ts_tablet_manager.cc:1428] T 31ed8480f8934264beb7733e3c767bea P 0d0ea86bb8464914b0cd04ed16aba960: Time spent starting tablet: real 0.015s user 0.008s sys 0.006s
I20260808 13:07:24.382869 26818 ts_tablet_manager.cc:1428] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537: Time spent starting tablet: real 0.015s user 0.008s sys 0.005s
W20260808 13:07:24.393605 26379 tablet.cc:2369] T 31ed8480f8934264beb7733e3c767bea P 766511b6ae0544c3a8eabf2ed87c344b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:07:24.433306 26582 tablet.cc:2369] T 31ed8480f8934264beb7733e3c767bea P 0d0ea86bb8464914b0cd04ed16aba960: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:07:24.436287 26830 raft_consensus.cc:482] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:07:24.436682 26830 raft_consensus.cc:504] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:07:24.439248 26830 leader_election.cc:288] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d0ea86bb8464914b0cd04ed16aba960 (127.1.124.130:46503), 766511b6ae0544c3a8eabf2ed87c344b (127.1.124.129:42343)
I20260808 13:07:24.449015 26791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:24.439351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:46503 (local address 127.1.124.131:35331)
0808 13:07:24.439861 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:24.439884 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:24.440028 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:24.440757 (+ 729us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:24.440766 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:24.440804 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:07:24.441043 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:24.441064 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.442361 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.442368 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:24.444695 (+ 2327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:24.444714 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.448106 (+ 3392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.448115 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:24.448359 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:24.448414 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 13:07:24.448512 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 13:07:24.448701 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":5}
I20260808 13:07:24.449378 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:24.441228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:48377 (local address 127.1.124.129:42343)
0808 13:07:24.441486 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:24.441495 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:24.441516 (+ 21us) server_negotiation.cc:407] Connection header received
0808 13:07:24.441595 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:24.441602 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:24.441683 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:24.441780 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:07:24.442440 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.443683 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.446608 (+ 2925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.448222 (+ 1614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.448359 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:24.448431 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:24.449002 (+ 571us) server_negotiation.cc:299] Negotiation successful
0808 13:07:24.449140 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 13:07:24.449664 26809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:24.440037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:35331 (local address 127.1.124.130:46503)
0808 13:07:24.440371 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:24.440379 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:24.440402 (+ 23us) server_negotiation.cc:407] Connection header received
0808 13:07:24.440486 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:24.440493 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:24.440578 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:24.440711 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:07:24.441232 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.442168 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.444904 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.446307 (+ 1403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.446474 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:24.446577 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:24.448837 (+ 2260us) server_negotiation.cc:299] Negotiation successful
0808 13:07:24.448977 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260808 13:07:24.450138 26834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:24.439877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:42343 (local address 127.1.124.131:48377)
0808 13:07:24.440728 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:24.440749 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:24.440961 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:24.441834 (+ 873us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:24.441839 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:24.441865 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:07:24.442221 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:24.442236 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.443910 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.443916 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:24.446366 (+ 2450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:24.446387 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.448639 (+ 2252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.448647 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:24.448805 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:24.448836 (+ 31us) client_negotiation.cc:769] Sending connection context
0808 13:07:24.449658 (+ 822us) client_negotiation.cc:240] Negotiation successful
0808 13:07:24.449835 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"mutex_wait_us":1,"thread_start_us":481,"threads_started":1}
I20260808 13:07:24.450793 26513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31ed8480f8934264beb7733e3c767bea" candidate_uuid: "9db875a520d44bb693c809b4a6ea9537" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d0ea86bb8464914b0cd04ed16aba960" is_pre_election: true
I20260808 13:07:24.451465 26513 raft_consensus.cc:2454] T 31ed8480f8934264beb7733e3c767bea P 0d0ea86bb8464914b0cd04ed16aba960 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9db875a520d44bb693c809b4a6ea9537 in term 0.
I20260808 13:07:24.451418 26311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31ed8480f8934264beb7733e3c767bea" candidate_uuid: "9db875a520d44bb693c809b4a6ea9537" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "766511b6ae0544c3a8eabf2ed87c344b" is_pre_election: true
I20260808 13:07:24.451979 26311 raft_consensus.cc:2454] T 31ed8480f8934264beb7733e3c767bea P 766511b6ae0544c3a8eabf2ed87c344b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9db875a520d44bb693c809b4a6ea9537 in term 0.
I20260808 13:07:24.452296 26621 leader_election.cc:302] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d0ea86bb8464914b0cd04ed16aba960, 9db875a520d44bb693c809b4a6ea9537; no voters:
I20260808 13:07:24.452683 26830 raft_consensus.cc:2781] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:07:24.452847 26830 raft_consensus.cc:482] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:07:24.452996 26830 raft_consensus.cc:3037] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:07:24.455260 26830 raft_consensus.cc:504] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:07:24.455879 26830 leader_election.cc:288] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537 [CANDIDATE]: Term 1 election: Requested vote from peers 0d0ea86bb8464914b0cd04ed16aba960 (127.1.124.130:46503), 766511b6ae0544c3a8eabf2ed87c344b (127.1.124.129:42343)
I20260808 13:07:24.456442 26513 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31ed8480f8934264beb7733e3c767bea" candidate_uuid: "9db875a520d44bb693c809b4a6ea9537" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d0ea86bb8464914b0cd04ed16aba960"
I20260808 13:07:24.456508 26311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "31ed8480f8934264beb7733e3c767bea" candidate_uuid: "9db875a520d44bb693c809b4a6ea9537" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "766511b6ae0544c3a8eabf2ed87c344b"
I20260808 13:07:24.456692 26513 raft_consensus.cc:3037] T 31ed8480f8934264beb7733e3c767bea P 0d0ea86bb8464914b0cd04ed16aba960 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:07:24.456712 26311 raft_consensus.cc:3037] T 31ed8480f8934264beb7733e3c767bea P 766511b6ae0544c3a8eabf2ed87c344b [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:07:24.459997 26311 raft_consensus.cc:2454] T 31ed8480f8934264beb7733e3c767bea P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9db875a520d44bb693c809b4a6ea9537 in term 1.
I20260808 13:07:24.459997 26513 raft_consensus.cc:2454] T 31ed8480f8934264beb7733e3c767bea P 0d0ea86bb8464914b0cd04ed16aba960 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9db875a520d44bb693c809b4a6ea9537 in term 1.
I20260808 13:07:24.460446 26621 leader_election.cc:302] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 766511b6ae0544c3a8eabf2ed87c344b, 9db875a520d44bb693c809b4a6ea9537; no voters:
I20260808 13:07:24.460757 26830 raft_consensus.cc:2781] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:07:24.461221 26830 raft_consensus.cc:686] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537 [term 1 LEADER]: Becoming Leader. State: Replica: 9db875a520d44bb693c809b4a6ea9537, State: Running, Role: LEADER
I20260808 13:07:24.462049 26830 consensus_queue.cc:237] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:07:24.467273 26090 catalog_manager.cc:5495] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9db875a520d44bb693c809b4a6ea9537 (127.1.124.131). New cstate: current_term: 1 leader_uuid: "9db875a520d44bb693c809b4a6ea9537" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } health_report { overall_health: UNKNOWN } } }
W20260808 13:07:24.482362 26787 tablet.cc:2369] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:07:24.491451 26810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:24.486096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57514 (local address 127.1.124.131:34113)
0808 13:07:24.486289 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:24.486293 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:24.486423 (+ 130us) server_negotiation.cc:407] Connection header received
0808 13:07:24.486645 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:24.486649 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:24.486725 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:24.486852 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:07:24.487222 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.487964 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.489508 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.489908 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.490002 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:24.490443 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:07:24.490958 (+ 515us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:07:24.491075 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:24.491188 (+ 113us) server_negotiation.cc:299] Negotiation successful
0808 13:07:24.491254 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 13:07:24.499862 26311 raft_consensus.cc:1261] T 31ed8480f8934264beb7733e3c767bea P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Refusing update from remote peer 9db875a520d44bb693c809b4a6ea9537: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:07:24.499862 26513 raft_consensus.cc:1261] T 31ed8480f8934264beb7733e3c767bea P 0d0ea86bb8464914b0cd04ed16aba960 [term 1 FOLLOWER]: Refusing update from remote peer 9db875a520d44bb693c809b4a6ea9537: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:07:24.500906 26836 consensus_queue.cc:1035] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:07:24.501327 26830 consensus_queue.cc:1035] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537 [LEADER]: Connected to new peer: Peer: permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:07:24.514082 26842 mvcc.cc:205] Tried to move back new op lower bound from 7316252444660228096 to 7316252444520968192. Current Snapshot: MvccSnapshot[applied={T|T < 7316252444660228096}]
I20260808 13:07:24.516785 26845 mvcc.cc:205] Tried to move back new op lower bound from 7316252444660228096 to 7316252444520968192. Current Snapshot: MvccSnapshot[applied={T|T < 7316252444660228096}]
I20260808 13:07:24.529626 26089 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:07:24.529942 26089 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45578:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:07:24.533740 26089 catalog_manager.cc:5782] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90: Sending DeleteTablet for 3 replicas of tablet 31ed8480f8934264beb7733e3c767bea
I20260808 13:07:24.534891 26481 tablet_service.cc:1514] Processing DeleteTablet for tablet 31ed8480f8934264beb7733e3c767bea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:07:24 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:59584
I20260808 13:07:24.534924 26683 tablet_service.cc:1514] Processing DeleteTablet for tablet 31ed8480f8934264beb7733e3c767bea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:07:24 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:57502
I20260808 13:07:24.535727 26282 tablet_service.cc:1514] Processing DeleteTablet for tablet 31ed8480f8934264beb7733e3c767bea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:07:24 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:48752
I20260808 13:07:24.535900 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 26192
I20260808 13:07:24.551930 26858 tablet_replica.cc:330] T 31ed8480f8934264beb7733e3c767bea P 0d0ea86bb8464914b0cd04ed16aba960: stopping tablet replica
I20260808 13:07:24.552393 26858 raft_consensus.cc:2229] T 31ed8480f8934264beb7733e3c767bea P 0d0ea86bb8464914b0cd04ed16aba960 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:07:24.552908 26858 raft_consensus.cc:2258] T 31ed8480f8934264beb7733e3c767bea P 0d0ea86bb8464914b0cd04ed16aba960 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:07:24.558071 26857 tablet_replica.cc:330] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537: stopping tablet replica
I20260808 13:07:24.558310 26858 ts_tablet_manager.cc:1905] T 31ed8480f8934264beb7733e3c767bea P 0d0ea86bb8464914b0cd04ed16aba960: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:07:24.558880 26859 tablet_replica.cc:330] T 31ed8480f8934264beb7733e3c767bea P 766511b6ae0544c3a8eabf2ed87c344b: stopping tablet replica
I20260808 13:07:24.559477 26859 raft_consensus.cc:2229] T 31ed8480f8934264beb7733e3c767bea P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:07:24.560142 26859 raft_consensus.cc:2258] T 31ed8480f8934264beb7733e3c767bea P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:07:24.560740 26857 raft_consensus.cc:2229] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:07:24.561534 26857 raft_consensus.cc:2258] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:07:24.562454 26859 ts_tablet_manager.cc:1905] T 31ed8480f8934264beb7733e3c767bea P 766511b6ae0544c3a8eabf2ed87c344b: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 13:07:24 dist-test-slave-t3q3 krb5kdc[26021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.129@KRBTEST.COM
I20260808 13:07:24.565721 26858 ts_tablet_manager.cc:1918] T 31ed8480f8934264beb7733e3c767bea P 0d0ea86bb8464914b0cd04ed16aba960: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:07:24.565991 26858 log.cc:1196] T 31ed8480f8934264beb7733e3c767bea P 0d0ea86bb8464914b0cd04ed16aba960: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/31ed8480f8934264beb7733e3c767bea
I20260808 13:07:24.566627 26858 ts_tablet_manager.cc:1939] T 31ed8480f8934264beb7733e3c767bea P 0d0ea86bb8464914b0cd04ed16aba960: Deleting consensus metadata
I20260808 13:07:24.566769 26857 ts_tablet_manager.cc:1905] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:07:24.568650 26073 catalog_manager.cc:4841] TS 0d0ea86bb8464914b0cd04ed16aba960 (127.1.124.130:46503): tablet 31ed8480f8934264beb7733e3c767bea (table test-table [id=baee8da3bd32425b8190b217a3e5bd00]) successfully deleted
I20260808 13:07:24.569840 26859 ts_tablet_manager.cc:1918] T 31ed8480f8934264beb7733e3c767bea P 766511b6ae0544c3a8eabf2ed87c344b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:07:24.570099 26859 log.cc:1196] T 31ed8480f8934264beb7733e3c767bea P 766511b6ae0544c3a8eabf2ed87c344b: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/31ed8480f8934264beb7733e3c767bea
I20260808 13:07:24.570701 26859 ts_tablet_manager.cc:1939] T 31ed8480f8934264beb7733e3c767bea P 766511b6ae0544c3a8eabf2ed87c344b: Deleting consensus metadata
I20260808 13:07:24.572765 26073 catalog_manager.cc:4841] TS 766511b6ae0544c3a8eabf2ed87c344b (127.1.124.129:42343): tablet 31ed8480f8934264beb7733e3c767bea (table test-table [id=baee8da3bd32425b8190b217a3e5bd00]) successfully deleted
I20260808 13:07:24.575418 26857 ts_tablet_manager.cc:1918] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:07:24.575603 26857 log.cc:1196] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/31ed8480f8934264beb7733e3c767bea
I20260808 13:07:24.576275 26857 ts_tablet_manager.cc:1939] T 31ed8480f8934264beb7733e3c767bea P 9db875a520d44bb693c809b4a6ea9537: Deleting consensus metadata
I20260808 13:07:24.577407 26812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:24.537473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48764 (local address 127.1.124.129:42343)
0808 13:07:24.537700 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:24.537709 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:24.547901 (+ 10192us) server_negotiation.cc:407] Connection header received
0808 13:07:24.548501 (+ 600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:24.548511 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:24.548621 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:24.548713 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:24.553384 (+ 4671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.554243 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.555789 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.556483 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.556591 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:24.569311 (+ 12720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:24.569326 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:24.569346 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:24.569400 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:24.573613 (+ 4213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:24.574457 (+ 844us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:24.574465 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:24.574467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:24.574599 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:24.575026 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:24.575033 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:24.575040 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:24.575470 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:24.575843 (+ 373us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:24.576602 (+ 759us) server_negotiation.cc:299] Negotiation successful
0808 13:07:24.576844 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 13:07:24.578130 26370 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:07:24.578207 26072 catalog_manager.cc:4841] TS 9db875a520d44bb693c809b4a6ea9537 (127.1.124.131:34113): tablet 31ed8480f8934264beb7733e3c767bea (table test-table [id=baee8da3bd32425b8190b217a3e5bd00]) successfully deleted
I20260808 13:07:24.701339 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 26192
I20260808 13:07:24.726118 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 26389
Aug 08 13:07:24 dist-test-slave-t3q3 krb5kdc[26021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.130@KRBTEST.COM
I20260808 13:07:24.746083 26809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:24.726966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59590 (local address 127.1.124.130:46503)
0808 13:07:24.727141 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:24.727148 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:24.727256 (+ 108us) server_negotiation.cc:407] Connection header received
0808 13:07:24.727468 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:24.727473 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:24.727558 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:24.727673 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:24.729668 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.730789 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.733153 (+ 2364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.733801 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.733949 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:24.739806 (+ 5857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:24.739823 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:24.739839 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:24.739880 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:24.742962 (+ 3082us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:24.743760 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:24.743768 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:24.743770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:24.743906 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:24.744428 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:24.744432 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:24.744434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:24.744825 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:24.745140 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:24.745505 (+ 365us) server_negotiation.cc:299] Negotiation successful
0808 13:07:24.745731 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 13:07:24.746788 26573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:07:24.849277 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 26389
I20260808 13:07:24.874595 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 26588
Aug 08 13:07:24 dist-test-slave-t3q3 krb5kdc[26021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194441, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.131@KRBTEST.COM
I20260808 13:07:24.893735 26810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:24.875446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57520 (local address 127.1.124.131:34113)
0808 13:07:24.875666 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:24.875673 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:24.875701 (+ 28us) server_negotiation.cc:407] Connection header received
0808 13:07:24.875873 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:24.875879 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:24.875994 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:24.876188 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:24.877609 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.878849 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.880389 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:24.881122 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:24.881292 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:24.885840 (+ 4548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:24.885860 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:24.885882 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:24.885936 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:24.890161 (+ 4225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:24.890960 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:24.890970 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:24.890974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:24.891217 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:24.891649 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:24.891655 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:24.891659 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:24.892196 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:24.892496 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:24.893006 (+ 510us) server_negotiation.cc:299] Negotiation successful
0808 13:07:24.893306 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 13:07:24.894560 26778 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:07:25.021298 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 26588
I20260808 13:07:25.049849 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 26041
I20260808 13:07:25.051139 26136 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:07:25.164770 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 26041
I20260808 13:07:45.197760 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:39149
--webserver_interface=127.1.124.190
--webserver_port=43803
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:45281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:39149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:07:45.547690 26866 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:07:45.548097 26866 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:07:45.548285 26866 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:07:45.555596 26866 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 13:07:45.555752 26866 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 13:07:45.555949 26866 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 13:07:45.556111 26866 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:07:45.556219 26866 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:07:45.556349 26866 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:07:45.556438 26866 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:07:45.556527 26866 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:07:45.566743 26866 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:45281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:39149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:39149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=43803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:07:45.568183 26866 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:07:45.569526 26866 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:07:45.595465 26878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:45.595506 26875 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:07:45.596148 26866 server_base.cc:1034] running on GCE node
W20260808 13:07:45.596210 26880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:07:45.597204 26866 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:07:45.598856 26866 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:07:45.600162 26866 hybrid_clock.cc:648] HybridClock initialized: now 1786194465600063 us; error 45 us; skew 500 ppm
I20260808 13:07:45.600593 26866 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:07:45 dist-test-slave-t3q3 krb5kdc[26021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194465, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:07:45.608860 26866 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:07:45.610662 26866 webserver.cc:460] Webserver started at http://127.1.124.190:43803/ using document root <none> and password file <none>
I20260808 13:07:45.611492 26866 fs_manager.cc:359] Metadata directory not provided
I20260808 13:07:45.611653 26866 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:07:45.618093 26866 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 13:07:45.621277 26887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:45.622794 26866 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:07:45.623010 26866 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "da2f58b07de549af9773d27e893bdb90"
format_stamp: "Formatted at 2026-08-08 13:07:22 on dist-test-slave-t3q3"
server_key: "bb983c92fb988c4423aef9a11fd102af"
server_key_iv: "47173ca5213611f84bf756385a9d7a55"
server_key_version: "encryptionkey@0"
I20260808 13:07:45.624132 26866 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:07:45.650162 26866 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:07:45.655015 26866 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:07:45.655354 26866 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:07:45.675910 26866 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:39149
I20260808 13:07:45.675951 26963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:39149 every 8 connection(s)
I20260808 13:07:45.677568 26866 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:07:45.679327 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 26866
I20260808 13:07:45.680305 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:42343
--local_ip_for_outbound_sockets=127.1.124.129
--tserver_master_addrs=127.1.124.190:39149
--webserver_port=46633
--webserver_interface=127.1.124.129
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:45281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:07:45.685086 26966 sys_catalog.cc:263] Verifying existing consensus state
I20260808 13:07:45.688623 26966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90: Bootstrap starting.
I20260808 13:07:45.708885 26966 log.cc:824] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90: Log is configured to *not* fsync() on all Append() calls
I20260808 13:07:45.721210 26966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 13:07:45.721801 26966 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90: Bootstrap complete.
I20260808 13:07:45.731453 26966 raft_consensus.cc:348] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2f58b07de549af9773d27e893bdb90" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39149 } }
I20260808 13:07:45.732430 26966 raft_consensus.cc:729] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: da2f58b07de549af9773d27e893bdb90, State: Initialized, Role: FOLLOWER
I20260808 13:07:45.733166 26966 consensus_queue.cc:260] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2f58b07de549af9773d27e893bdb90" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39149 } }
I20260808 13:07:45.733420 26966 raft_consensus.cc:388] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:07:45.733543 26966 raft_consensus.cc:482] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:07:45.733747 26966 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:07:45.737735 26966 raft_consensus.cc:504] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2f58b07de549af9773d27e893bdb90" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39149 } }
I20260808 13:07:45.738339 26966 leader_election.cc:302] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da2f58b07de549af9773d27e893bdb90; no voters:
I20260808 13:07:45.739049 26966 leader_election.cc:288] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 13:07:45.739244 26974 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 13:07:45.740460 26974 raft_consensus.cc:686] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 2 LEADER]: Becoming Leader. State: Replica: da2f58b07de549af9773d27e893bdb90, State: Running, Role: LEADER
I20260808 13:07:45.741096 26974 consensus_queue.cc:237] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2f58b07de549af9773d27e893bdb90" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39149 } }
I20260808 13:07:45.741689 26966 sys_catalog.cc:564] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:07:45.744942 26977 sys_catalog.cc:455] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "da2f58b07de549af9773d27e893bdb90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2f58b07de549af9773d27e893bdb90" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39149 } } }
I20260808 13:07:45.745246 26978 sys_catalog.cc:455] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da2f58b07de549af9773d27e893bdb90. Latest consensus state: current_term: 2 leader_uuid: "da2f58b07de549af9773d27e893bdb90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2f58b07de549af9773d27e893bdb90" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39149 } } }
I20260808 13:07:45.745658 26977 sys_catalog.cc:458] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [sys.catalog]: This master's current role is: LEADER
I20260808 13:07:45.745656 26978 sys_catalog.cc:458] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [sys.catalog]: This master's current role is: LEADER
I20260808 13:07:45.758543 26990 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:07:45.767006 26990 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:07:45.768534 26990 catalog_manager.cc:1259] Loaded cluster ID: 3d8168421f97430abc3d8d1aa248360a
I20260808 13:07:45.768711 26990 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:07:45.773170 26990 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:07:45.774981 26990 catalog_manager.cc:5879] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90: Loaded TSK: 0
I20260808 13:07:45.809384 26990 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90: Generated new TSK 1
I20260808 13:07:45.809830 26990 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 13:07:46.062343 26970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:07:46.062623 26970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:07:46.062773 26970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:07:46.070308 26970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:07:46.070437 26970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:07:46.070612 26970 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:07:46.080696 26970 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:45281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:42343
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=46633
--tserver_master_addrs=127.1.124.190:39149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:07:46.082057 26970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:07:46.083410 26970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:07:46.098215 27011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:46.098260 27012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:46.099134 27014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:07:46.100450 26970 server_base.cc:1034] running on GCE node
I20260808 13:07:46.101202 26970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:07:46.102240 26970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:07:46.103483 26970 hybrid_clock.cc:648] HybridClock initialized: now 1786194466103423 us; error 52 us; skew 500 ppm
I20260808 13:07:46.103847 26970 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:07:46 dist-test-slave-t3q3 krb5kdc[26021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194466, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:07:46.107205 26970 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:07:46.108850 26970 webserver.cc:460] Webserver started at http://127.1.124.129:46633/ using document root <none> and password file <none>
I20260808 13:07:46.109587 26970 fs_manager.cc:359] Metadata directory not provided
I20260808 13:07:46.109731 26970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:07:46.115444 26970 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 13:07:46.118525 27027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:46.120028 26970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:07:46.120308 26970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "766511b6ae0544c3a8eabf2ed87c344b"
format_stamp: "Formatted at 2026-08-08 13:07:23 on dist-test-slave-t3q3"
server_key: "47193f2a5ab60fb8c51e9a26256d237b"
server_key_iv: "eac434ea418f52172c97240e134831c3"
server_key_version: "encryptionkey@0"
I20260808 13:07:46.121353 26970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:07:46.143887 26970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:07:46.148532 26970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:07:46.148826 26970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:07:46.149935 26970 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:07:46.151341 26970 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:07:46.151475 26970 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:46.151615 26970 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:07:46.151705 26970 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:46.185756 26970 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:42343
I20260808 13:07:46.185850 27210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:42343 every 8 connection(s)
I20260808 13:07:46.187291 26970 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:07:46.193135 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 26970
I20260808 13:07:46.194015 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:46503
--local_ip_for_outbound_sockets=127.1.124.130
--tserver_master_addrs=127.1.124.190:39149
--webserver_port=34421
--webserver_interface=127.1.124.130
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:45281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 13:07:46 dist-test-slave-t3q3 krb5kdc[26021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194466, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:07:46.214464 27214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:46.190627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.129:47487)
0808 13:07:46.191348 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:46.191391 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:46.193740 (+ 2349us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:46.195190 (+ 1450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:46.195203 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:46.196028 (+ 825us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:07:46.196871 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:46.196899 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:46.198638 (+ 1739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:46.198646 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:46.200911 (+ 2265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:46.200931 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:46.202078 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:46.202088 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:46.202411 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:46.203204 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:07:46.203245 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:07:46.206023 (+ 2778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:07:46.209722 (+ 3699us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:07:46.209732 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:07:46.209755 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:07:46.210186 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:07:46.210738 (+ 552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:07:46.210746 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:07:46.210752 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:07:46.210947 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:07:46.211715 (+ 768us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:07:46.211723 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:07:46.212215 (+ 492us) client_negotiation.cc:769] Sending connection context
0808 13:07:46.212491 (+ 276us) client_negotiation.cc:240] Negotiation successful
0808 13:07:46.213087 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":266,"threads_started":1}
I20260808 13:07:46.214464 27213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:46.190290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:47487 (local address 127.1.124.190:39149)
0808 13:07:46.190854 (+ 564us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:46.190866 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:46.191528 (+ 662us) server_negotiation.cc:407] Connection header received
0808 13:07:46.194349 (+ 2821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:46.194361 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:46.194679 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:46.194964 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:46.197094 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:46.198431 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:46.201088 (+ 2657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:46.201811 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:46.201973 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:46.206350 (+ 4377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:46.206358 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:46.206374 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:46.206413 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:46.209562 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:46.210349 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:46.210357 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:46.210364 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:46.210578 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:46.211079 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:46.211084 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:46.211093 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:46.211561 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:46.211793 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:46.212722 (+ 929us) server_negotiation.cc:299] Negotiation successful
0808 13:07:46.213144 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":248,"threads_started":1}
I20260808 13:07:46.218070 27211 heartbeater.cc:347] Connected to a master server at 127.1.124.190:39149
I20260808 13:07:46.218430 27211 heartbeater.cc:464] Registering TS with master...
I20260808 13:07:46.219187 27211 heartbeater.cc:511] Master 127.1.124.190:39149 requested a full tablet report, sending...
I20260808 13:07:46.221386 26911 ts_manager.cc:194] Registered new tserver with Master: 766511b6ae0544c3a8eabf2ed87c344b (127.1.124.129:42343)
I20260808 13:07:46.223279 26911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:47487
W20260808 13:07:46.544991 27215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:07:46.545287 27215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:07:46.545455 27215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:07:46.552685 27215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:07:46.552798 27215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:07:46.553013 27215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:07:46.562775 27215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:45281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:46503
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=34421
--tserver_master_addrs=127.1.124.190:39149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:07:46.564126 27215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:07:46.565569 27215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:07:46.579187 27223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:46.579311 27222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:46.579278 27226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:07:46.580318 27215 server_base.cc:1034] running on GCE node
I20260808 13:07:46.581017 27215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:07:46.581955 27215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:07:46.583159 27215 hybrid_clock.cc:648] HybridClock initialized: now 1786194466583082 us; error 66 us; skew 500 ppm
I20260808 13:07:46.583448 27215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:07:46 dist-test-slave-t3q3 krb5kdc[26021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194466, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:07:46.586809 27215 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:07:46.588754 27215 webserver.cc:460] Webserver started at http://127.1.124.130:34421/ using document root <none> and password file <none>
I20260808 13:07:46.589463 27215 fs_manager.cc:359] Metadata directory not provided
I20260808 13:07:46.589639 27215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:07:46.595328 27215 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.006s
I20260808 13:07:46.598326 27236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:46.599741 27215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 13:07:46.599961 27215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "0d0ea86bb8464914b0cd04ed16aba960"
format_stamp: "Formatted at 2026-08-08 13:07:23 on dist-test-slave-t3q3"
server_key: "ad2b3387acab3f88f20938211d1f38b1"
server_key_iv: "debb18a0e740dc23c6f066a4d24f60b8"
server_key_version: "encryptionkey@0"
I20260808 13:07:46.601109 27215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:07:46.623456 27215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:07:46.628417 27215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:07:46.628724 27215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:07:46.629796 27215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:07:46.631162 27215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:07:46.631253 27215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:46.631340 27215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:07:46.631402 27215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:46.666821 27215 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:46503
I20260808 13:07:46.666867 27411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:46503 every 8 connection(s)
I20260808 13:07:46.668380 27215 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:07:46.677748 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 27215
I20260808 13:07:46.678414 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:34113
--local_ip_for_outbound_sockets=127.1.124.131
--tserver_master_addrs=127.1.124.190:39149
--webserver_port=39963
--webserver_interface=127.1.124.131
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:45281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 13:07:46 dist-test-slave-t3q3 krb5kdc[26021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194466, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:07:46.693341 27213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:46.671187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:46103 (local address 127.1.124.190:39149)
0808 13:07:46.671396 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:46.671400 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:46.672751 (+ 1351us) server_negotiation.cc:407] Connection header received
0808 13:07:46.674742 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:46.674748 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:46.674840 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:46.674954 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:46.676676 (+ 1722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:46.677781 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:46.680209 (+ 2428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:46.681016 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:46.681171 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:46.685898 (+ 4727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:46.685910 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:46.685916 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:46.685956 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:46.689526 (+ 3570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:46.690535 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:46.690542 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:46.690545 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:46.690714 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:46.691292 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:46.691297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:46.691301 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:46.691663 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:46.691806 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:46.692688 (+ 882us) server_negotiation.cc:299] Negotiation successful
0808 13:07:46.692929 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 13:07:46.694370 27415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:46.671709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.130:46103)
0808 13:07:46.672515 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:46.672609 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:46.674458 (+ 1849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:46.675224 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:46.675232 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:46.675737 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:07:46.676477 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:46.676502 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:46.677993 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:46.677999 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:46.680001 (+ 2002us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:46.680016 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:46.681285 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:46.681294 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:46.681588 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:46.682392 (+ 804us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:07:46.682427 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:07:46.685581 (+ 3154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:07:46.689688 (+ 4107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:07:46.689700 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:07:46.689720 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:07:46.690385 (+ 665us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:07:46.690856 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:07:46.690863 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:07:46.690867 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:07:46.691138 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:07:46.691814 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:07:46.691825 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:07:46.692272 (+ 447us) client_negotiation.cc:769] Sending connection context
0808 13:07:46.692583 (+ 311us) client_negotiation.cc:240] Negotiation successful
0808 13:07:46.693126 (+ 543us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":365,"threads_started":1}
I20260808 13:07:46.695886 27412 heartbeater.cc:347] Connected to a master server at 127.1.124.190:39149
I20260808 13:07:46.696247 27412 heartbeater.cc:464] Registering TS with master...
I20260808 13:07:46.696972 27412 heartbeater.cc:511] Master 127.1.124.190:39149 requested a full tablet report, sending...
I20260808 13:07:46.698432 26911 ts_manager.cc:194] Registered new tserver with Master: 0d0ea86bb8464914b0cd04ed16aba960 (127.1.124.130:46503)
I20260808 13:07:46.699278 26911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:46103
W20260808 13:07:47.040017 27416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:07:47.040365 27416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:07:47.040551 27416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:07:47.047869 27416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:07:47.047969 27416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:07:47.048262 27416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:07:47.057804 27416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:45281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:34113
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=39963
--tserver_master_addrs=127.1.124.190:39149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:07:47.059060 27416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:07:47.060529 27416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:07:47.073732 27422 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:47.073736 27424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:07:47.073776 27426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:07:47.074898 27416 server_base.cc:1034] running on GCE node
I20260808 13:07:47.075569 27416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:07:47.076617 27416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:07:47.077865 27416 hybrid_clock.cc:648] HybridClock initialized: now 1786194467077803 us; error 54 us; skew 500 ppm
I20260808 13:07:47.078233 27416 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:07:47 dist-test-slave-t3q3 krb5kdc[26021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194467, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:07:47.081655 27416 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:07:47.083189 27416 webserver.cc:460] Webserver started at http://127.1.124.131:39963/ using document root <none> and password file <none>
I20260808 13:07:47.083879 27416 fs_manager.cc:359] Metadata directory not provided
I20260808 13:07:47.083997 27416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:07:47.090049 27416 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.004s
I20260808 13:07:47.093086 27440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:47.094547 27416 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:07:47.094727 27416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "9db875a520d44bb693c809b4a6ea9537"
format_stamp: "Formatted at 2026-08-08 13:07:24 on dist-test-slave-t3q3"
server_key: "11977007e82c74b72c46429bb4df8250"
server_key_iv: "25a660ee84e4f7154333d89d115916d3"
server_key_version: "encryptionkey@0"
I20260808 13:07:47.095744 27416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:07:47.131795 27416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:07:47.137076 27416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:07:47.137451 27416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:07:47.138808 27416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:07:47.140281 27416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:07:47.140407 27416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:47.140539 27416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:07:47.140638 27416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:07:47.177347 27416 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:34113
I20260808 13:07:47.177439 27613 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:34113 every 8 connection(s)
I20260808 13:07:47.179025 27416 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:07:47.184876 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 27416
Aug 08 13:07:47 dist-test-slave-t3q3 krb5kdc[26021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194467, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:07:47.207397 27213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.181811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:50955 (local address 127.1.124.190:39149)
0808 13:07:47.182012 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:47.182018 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:47.183341 (+ 1323us) server_negotiation.cc:407] Connection header received
0808 13:07:47.185413 (+ 2072us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:47.185418 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:47.185490 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:47.185608 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:47.188179 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.189711 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.193597 (+ 3886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.194529 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.194653 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:47.200317 (+ 5664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:47.200326 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:47.200334 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:47.200382 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:47.203944 (+ 3562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:47.204874 (+ 930us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:47.204881 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:47.204883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:47.205007 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:47.205535 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:47.205540 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:47.205545 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:47.205881 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:47.205993 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:47.206970 (+ 977us) server_negotiation.cc:299] Negotiation successful
0808 13:07:47.207141 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 13:07:47.209105 27617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.182379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.131:50955)
0808 13:07:47.183148 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:47.183188 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:47.184963 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:47.185885 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:47.185898 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:47.186828 (+ 930us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:07:47.187844 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:47.187877 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.190026 (+ 2149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.190035 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:47.193361 (+ 3326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:47.193385 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.195112 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.195122 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:47.195528 (+ 406us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:47.196458 (+ 930us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:07:47.196507 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:07:47.200022 (+ 3515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:07:47.204135 (+ 4113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:07:47.204142 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:07:47.204156 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:07:47.204685 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:07:47.205130 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:07:47.205136 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:07:47.205140 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:07:47.205403 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:07:47.206027 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:07:47.206036 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:07:47.206519 (+ 483us) client_negotiation.cc:769] Sending connection context
0808 13:07:47.206886 (+ 367us) client_negotiation.cc:240] Negotiation successful
0808 13:07:47.207497 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":293,"threads_started":1}
I20260808 13:07:47.210805 27615 heartbeater.cc:347] Connected to a master server at 127.1.124.190:39149
I20260808 13:07:47.211165 27615 heartbeater.cc:464] Registering TS with master...
I20260808 13:07:47.211925 27615 heartbeater.cc:511] Master 127.1.124.190:39149 requested a full tablet report, sending...
I20260808 13:07:47.213549 26911 ts_manager.cc:194] Registered new tserver with Master: 9db875a520d44bb693c809b4a6ea9537 (127.1.124.131:34113)
I20260808 13:07:47.214393 26911 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:50955
I20260808 13:07:47.215022 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.186040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43082 (local address 127.1.124.190:39149)
0808 13:07:47.198727 (+ 12687us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:47.198734 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:47.199141 (+ 407us) server_negotiation.cc:407] Connection header received
0808 13:07:47.199333 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:47.199339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:47.199427 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:47.199548 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:47.201735 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.202783 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.204807 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.205393 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.205546 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:47.208999 (+ 3453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:47.209003 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:47.209006 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:47.209034 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:47.212058 (+ 3024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:47.212794 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:47.212798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:47.212801 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:47.212913 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:47.213381 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:47.213393 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:47.213396 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:47.213748 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:47.214029 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:47.214361 (+ 332us) server_negotiation.cc:299] Negotiation successful
0808 13:07:47.214588 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12541,"spinlock_wait_cycles":5504,"thread_start_us":12404,"threads_started":1}
I20260808 13:07:47.217501 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:07:47.224483 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.218366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43098 (local address 127.1.124.190:39149)
0808 13:07:47.218534 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:47.218538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:47.219029 (+ 491us) server_negotiation.cc:407] Connection header received
0808 13:07:47.219283 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:47.219287 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:47.219424 (+ 137us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:47.219589 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:07:47.220023 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.220773 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.222332 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.222763 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.222902 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:47.223466 (+ 564us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:07:47.223742 (+ 276us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 13:07:47.224179 (+ 437us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43098: token expired
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 13:07:47.224642 27620 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43098: token expired
W20260808 13:07:47.225176 26795 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.1.124.190:39149)
I20260808 13:07:47.226558 27211 heartbeater.cc:503] Master 127.1.124.190:39149 was elected leader, sending a full tablet report...
I20260808 13:07:47.239143 27620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.225719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43112 (local address 127.1.124.190:39149)
0808 13:07:47.225879 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:47.225883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:47.225976 (+ 93us) server_negotiation.cc:407] Connection header received
0808 13:07:47.226096 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:47.226099 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:47.226158 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:47.226234 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:47.228040 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.228896 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.230367 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.230838 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.230950 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:47.233866 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:47.233871 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:47.233875 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:47.233905 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:47.236658 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:47.237311 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:47.237314 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:47.237317 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:47.237445 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:47.237778 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:47.237781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:47.237783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:47.238119 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:47.238298 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:47.238650 (+ 352us) server_negotiation.cc:299] Negotiation successful
0808 13:07:47.238859 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 13:07:47.242926 26911 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43112:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:07:47.250392 26911 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:07:47.272735 27628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.262005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:46503 (local address 127.0.0.1:40674)
0808 13:07:47.263405 (+ 1400us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:47.263447 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:47.263639 (+ 192us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:47.264418 (+ 779us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:47.264425 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:47.264455 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:07:47.264939 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:47.264954 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.267048 (+ 2094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.267053 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:47.270154 (+ 3101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:47.270167 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.271615 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.271621 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:47.271821 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:47.271877 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 13:07:47.272017 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 13:07:47.272320 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1101,"mutex_wait_us":5,"thread_start_us":915,"threads_started":1}
I20260808 13:07:47.272815 27629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.262182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40674 (local address 127.1.124.130:46503)
0808 13:07:47.262772 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:47.262783 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:47.263550 (+ 767us) server_negotiation.cc:407] Connection header received
0808 13:07:47.263866 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:47.263877 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:47.264200 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:47.264404 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:07:47.265144 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.266904 (+ 1760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.270310 (+ 3406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.271394 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.271537 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:47.271762 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:47.272302 (+ 540us) server_negotiation.cc:299] Negotiation successful
0808 13:07:47.272477 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":395,"threads_started":1}
I20260808 13:07:47.273955 27630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.262408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:34113 (local address 127.0.0.1:40478)
0808 13:07:47.263416 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:47.263446 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:47.263639 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:47.265901 (+ 2262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:47.265906 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:47.265931 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:07:47.266351 (+ 420us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:47.266365 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.268933 (+ 2568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.268938 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:47.271199 (+ 2261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:47.271213 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.273103 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.273109 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:47.273283 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:47.273319 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 13:07:47.273403 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 13:07:47.273543 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":894,"mutex_wait_us":21,"thread_start_us":828,"threads_started":1}
I20260808 13:07:47.274333 27633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.264373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:42343 (local address 127.0.0.1:53456)
0808 13:07:47.265518 (+ 1145us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:47.265536 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:47.265664 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:47.266340 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:47.266344 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:47.266370 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:07:47.266782 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:47.266797 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.268629 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.268636 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:47.271052 (+ 2416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:47.271075 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.273475 (+ 2400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.273481 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:47.273687 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:47.273720 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 13:07:47.273812 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 13:07:47.273956 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1026,"mutex_wait_us":84,"thread_start_us":882,"threads_started":1}
I20260808 13:07:47.275038 27632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.264217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53456 (local address 127.1.124.129:42343)
0808 13:07:47.265262 (+ 1045us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:47.265273 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:47.265784 (+ 511us) server_negotiation.cc:407] Connection header received
0808 13:07:47.265885 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:47.265893 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:47.266111 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:47.266300 (+ 189us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:07:47.266951 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.268484 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.271240 (+ 2756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.272847 (+ 1607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.273022 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:47.273272 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:47.274543 (+ 1271us) server_negotiation.cc:299] Negotiation successful
0808 13:07:47.274714 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":791,"thread_start_us":646,"threads_started":1}
I20260808 13:07:47.275243 27631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.263094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40478 (local address 127.1.124.131:34113)
0808 13:07:47.265310 (+ 2216us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:47.265330 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:47.265368 (+ 38us) server_negotiation.cc:407] Connection header received
0808 13:07:47.265469 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:47.265476 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:47.265692 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:47.266622 (+ 930us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:07:47.267041 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.268792 (+ 1751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.271344 (+ 2552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.272825 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.274421 (+ 1596us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:47.274652 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:47.274738 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 13:07:47.274917 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1045,"thread_start_us":863,"threads_started":1}
I20260808 13:07:47.277613 27314 tablet_service.cc:1467] Processing CreateTablet for tablet 19f0115622d745a6b331dbd395e6d116 (DEFAULT_TABLE table=test-table [id=4f1626dba6644d65a4453ffbf5666085]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:07:47.279034 27314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f0115622d745a6b331dbd395e6d116. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:07:47.278576 27106 tablet_service.cc:1467] Processing CreateTablet for tablet 19f0115622d745a6b331dbd395e6d116 (DEFAULT_TABLE table=test-table [id=4f1626dba6644d65a4453ffbf5666085]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:07:47.278566 27518 tablet_service.cc:1467] Processing CreateTablet for tablet 19f0115622d745a6b331dbd395e6d116 (DEFAULT_TABLE table=test-table [id=4f1626dba6644d65a4453ffbf5666085]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:07:47.279963 27518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f0115622d745a6b331dbd395e6d116. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:07:47.279974 27106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f0115622d745a6b331dbd395e6d116. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:07:47.291189 27635 tablet_bootstrap.cc:492] T 19f0115622d745a6b331dbd395e6d116 P 0d0ea86bb8464914b0cd04ed16aba960: Bootstrap starting.
I20260808 13:07:47.295805 27635 tablet_bootstrap.cc:654] T 19f0115622d745a6b331dbd395e6d116 P 0d0ea86bb8464914b0cd04ed16aba960: Neither blocks nor log segments found. Creating new log.
I20260808 13:07:47.296898 27639 tablet_bootstrap.cc:492] T 19f0115622d745a6b331dbd395e6d116 P 9db875a520d44bb693c809b4a6ea9537: Bootstrap starting.
I20260808 13:07:47.297300 27635 log.cc:824] T 19f0115622d745a6b331dbd395e6d116 P 0d0ea86bb8464914b0cd04ed16aba960: Log is configured to *not* fsync() on all Append() calls
I20260808 13:07:47.297746 27636 tablet_bootstrap.cc:492] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b: Bootstrap starting.
I20260808 13:07:47.301136 27635 tablet_bootstrap.cc:492] T 19f0115622d745a6b331dbd395e6d116 P 0d0ea86bb8464914b0cd04ed16aba960: No bootstrap required, opened a new log
I20260808 13:07:47.301416 27639 tablet_bootstrap.cc:654] T 19f0115622d745a6b331dbd395e6d116 P 9db875a520d44bb693c809b4a6ea9537: Neither blocks nor log segments found. Creating new log.
I20260808 13:07:47.301443 27635 ts_tablet_manager.cc:1397] T 19f0115622d745a6b331dbd395e6d116 P 0d0ea86bb8464914b0cd04ed16aba960: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260808 13:07:47.301570 27636 tablet_bootstrap.cc:654] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b: Neither blocks nor log segments found. Creating new log.
I20260808 13:07:47.302870 27639 log.cc:824] T 19f0115622d745a6b331dbd395e6d116 P 9db875a520d44bb693c809b4a6ea9537: Log is configured to *not* fsync() on all Append() calls
I20260808 13:07:47.302929 27636 log.cc:824] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b: Log is configured to *not* fsync() on all Append() calls
I20260808 13:07:47.306039 27636 tablet_bootstrap.cc:492] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b: No bootstrap required, opened a new log
I20260808 13:07:47.306210 27639 tablet_bootstrap.cc:492] T 19f0115622d745a6b331dbd395e6d116 P 9db875a520d44bb693c809b4a6ea9537: No bootstrap required, opened a new log
I20260808 13:07:47.306344 27636 ts_tablet_manager.cc:1397] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260808 13:07:47.306478 27639 ts_tablet_manager.cc:1397] T 19f0115622d745a6b331dbd395e6d116 P 9db875a520d44bb693c809b4a6ea9537: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260808 13:07:47.313915 27635 raft_consensus.cc:348] T 19f0115622d745a6b331dbd395e6d116 P 0d0ea86bb8464914b0cd04ed16aba960 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:07:47.314504 27635 raft_consensus.cc:374] T 19f0115622d745a6b331dbd395e6d116 P 0d0ea86bb8464914b0cd04ed16aba960 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:07:47.314635 27635 raft_consensus.cc:729] T 19f0115622d745a6b331dbd395e6d116 P 0d0ea86bb8464914b0cd04ed16aba960 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d0ea86bb8464914b0cd04ed16aba960, State: Initialized, Role: FOLLOWER
I20260808 13:07:47.315328 27635 consensus_queue.cc:260] T 19f0115622d745a6b331dbd395e6d116 P 0d0ea86bb8464914b0cd04ed16aba960 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:07:47.317617 27636 raft_consensus.cc:348] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:07:47.318136 27636 raft_consensus.cc:374] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:07:47.318238 27636 raft_consensus.cc:729] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 766511b6ae0544c3a8eabf2ed87c344b, State: Initialized, Role: FOLLOWER
I20260808 13:07:47.318930 27636 consensus_queue.cc:260] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:07:47.320866 27636 ts_tablet_manager.cc:1428] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260808 13:07:47.322099 27639 raft_consensus.cc:348] T 19f0115622d745a6b331dbd395e6d116 P 9db875a520d44bb693c809b4a6ea9537 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:07:47.322551 27639 raft_consensus.cc:374] T 19f0115622d745a6b331dbd395e6d116 P 9db875a520d44bb693c809b4a6ea9537 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:07:47.322672 27639 raft_consensus.cc:729] T 19f0115622d745a6b331dbd395e6d116 P 9db875a520d44bb693c809b4a6ea9537 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9db875a520d44bb693c809b4a6ea9537, State: Initialized, Role: FOLLOWER
I20260808 13:07:47.323299 27639 consensus_queue.cc:260] T 19f0115622d745a6b331dbd395e6d116 P 9db875a520d44bb693c809b4a6ea9537 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:07:47.324617 27615 heartbeater.cc:503] Master 127.1.124.190:39149 was elected leader, sending a full tablet report...
I20260808 13:07:47.329741 27635 ts_tablet_manager.cc:1428] T 19f0115622d745a6b331dbd395e6d116 P 0d0ea86bb8464914b0cd04ed16aba960: Time spent starting tablet: real 0.028s user 0.020s sys 0.000s
I20260808 13:07:47.330176 27412 heartbeater.cc:503] Master 127.1.124.190:39149 was elected leader, sending a full tablet report...
I20260808 13:07:47.332760 27639 ts_tablet_manager.cc:1428] T 19f0115622d745a6b331dbd395e6d116 P 9db875a520d44bb693c809b4a6ea9537: Time spent starting tablet: real 0.026s user 0.015s sys 0.000s
W20260808 13:07:47.420573 27413 tablet.cc:2369] T 19f0115622d745a6b331dbd395e6d116 P 0d0ea86bb8464914b0cd04ed16aba960: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:07:47.430869 27616 tablet.cc:2369] T 19f0115622d745a6b331dbd395e6d116 P 9db875a520d44bb693c809b4a6ea9537: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:07:47.442853 27646 raft_consensus.cc:482] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:07:47.443198 27646 raft_consensus.cc:504] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:07:47.445050 27646 leader_election.cc:288] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d0ea86bb8464914b0cd04ed16aba960 (127.1.124.130:46503), 9db875a520d44bb693c809b4a6ea9537 (127.1.124.131:34113)
W20260808 13:07:47.446605 27212 tablet.cc:2369] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:07:47.453029 27648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.445599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:46503 (local address 127.1.124.129:51543)
0808 13:07:47.446532 (+ 933us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:47.446552 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:47.446702 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:47.447058 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:47.447062 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:47.447089 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:07:47.447555 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:47.447573 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.448835 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.448838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:47.450880 (+ 2042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:47.450895 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.452368 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.452372 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:47.452489 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:47.452514 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 13:07:47.452591 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 13:07:47.452676 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":799,"thread_start_us":584,"threads_started":1}
I20260808 13:07:47.453107 27629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.445557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:51543 (local address 127.1.124.130:46503)
0808 13:07:47.445795 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:47.445801 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:47.446677 (+ 876us) server_negotiation.cc:407] Connection header received
0808 13:07:47.446875 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:47.446879 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:47.446938 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:47.447037 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:07:47.447735 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.448702 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.451054 (+ 2352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.452178 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.452338 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:47.452414 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:47.452654 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 13:07:47.452787 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 13:07:47.453809 27631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.445568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:51573 (local address 127.1.124.131:34113)
0808 13:07:47.445788 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:47.445795 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:47.447592 (+ 1797us) server_negotiation.cc:407] Connection header received
0808 13:07:47.447752 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:47.447757 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:47.447841 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:47.447948 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:07:47.448501 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.449327 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.451520 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.452790 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.452955 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:47.453037 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:47.453472 (+ 435us) server_negotiation.cc:299] Negotiation successful
0808 13:07:47.453619 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 13:07:47.453850 27649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.445643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:34113 (local address 127.1.124.129:51573)
0808 13:07:47.447460 (+ 1817us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:47.447483 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:07:47.447619 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:07:47.447965 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:07:47.447968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:07:47.447986 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:07:47.448336 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:47.448345 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.449463 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.449467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:47.451358 (+ 1891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:07:47.451369 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.453046 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.453056 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:07:47.453249 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:47.453297 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 13:07:47.453430 (+ 133us) client_negotiation.cc:240] Negotiation successful
0808 13:07:47.453518 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1676,"thread_start_us":538,"threads_started":1}
I20260808 13:07:47.454164 27341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f0115622d745a6b331dbd395e6d116" candidate_uuid: "766511b6ae0544c3a8eabf2ed87c344b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d0ea86bb8464914b0cd04ed16aba960" is_pre_election: true
I20260808 13:07:47.454666 27341 raft_consensus.cc:2454] T 19f0115622d745a6b331dbd395e6d116 P 0d0ea86bb8464914b0cd04ed16aba960 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 766511b6ae0544c3a8eabf2ed87c344b in term 0.
I20260808 13:07:47.454785 27543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f0115622d745a6b331dbd395e6d116" candidate_uuid: "766511b6ae0544c3a8eabf2ed87c344b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9db875a520d44bb693c809b4a6ea9537" is_pre_election: true
I20260808 13:07:47.455361 27543 raft_consensus.cc:2454] T 19f0115622d745a6b331dbd395e6d116 P 9db875a520d44bb693c809b4a6ea9537 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 766511b6ae0544c3a8eabf2ed87c344b in term 0.
I20260808 13:07:47.455416 27032 leader_election.cc:302] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d0ea86bb8464914b0cd04ed16aba960, 766511b6ae0544c3a8eabf2ed87c344b; no voters:
I20260808 13:07:47.455852 27646 raft_consensus.cc:2781] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:07:47.456022 27646 raft_consensus.cc:482] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:07:47.456193 27646 raft_consensus.cc:3037] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:07:47.458431 27646 raft_consensus.cc:504] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:07:47.459065 27646 leader_election.cc:288] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 1 election: Requested vote from peers 0d0ea86bb8464914b0cd04ed16aba960 (127.1.124.130:46503), 9db875a520d44bb693c809b4a6ea9537 (127.1.124.131:34113)
I20260808 13:07:47.459481 27341 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f0115622d745a6b331dbd395e6d116" candidate_uuid: "766511b6ae0544c3a8eabf2ed87c344b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d0ea86bb8464914b0cd04ed16aba960"
I20260808 13:07:47.459502 27543 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f0115622d745a6b331dbd395e6d116" candidate_uuid: "766511b6ae0544c3a8eabf2ed87c344b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9db875a520d44bb693c809b4a6ea9537"
I20260808 13:07:47.459700 27341 raft_consensus.cc:3037] T 19f0115622d745a6b331dbd395e6d116 P 0d0ea86bb8464914b0cd04ed16aba960 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:07:47.459697 27543 raft_consensus.cc:3037] T 19f0115622d745a6b331dbd395e6d116 P 9db875a520d44bb693c809b4a6ea9537 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:07:47.462045 27341 raft_consensus.cc:2454] T 19f0115622d745a6b331dbd395e6d116 P 0d0ea86bb8464914b0cd04ed16aba960 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 766511b6ae0544c3a8eabf2ed87c344b in term 1.
I20260808 13:07:47.462074 27543 raft_consensus.cc:2454] T 19f0115622d745a6b331dbd395e6d116 P 9db875a520d44bb693c809b4a6ea9537 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 766511b6ae0544c3a8eabf2ed87c344b in term 1.
I20260808 13:07:47.462451 27030 leader_election.cc:302] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 766511b6ae0544c3a8eabf2ed87c344b, 9db875a520d44bb693c809b4a6ea9537; no voters:
I20260808 13:07:47.462743 27646 raft_consensus.cc:2781] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:07:47.463281 27646 raft_consensus.cc:686] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 1 LEADER]: Becoming Leader. State: Replica: 766511b6ae0544c3a8eabf2ed87c344b, State: Running, Role: LEADER
I20260808 13:07:47.463866 27646 consensus_queue.cc:237] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:07:47.469051 26911 catalog_manager.cc:5495] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b reported cstate change: term changed from 0 to 1, leader changed from <none> to 766511b6ae0544c3a8eabf2ed87c344b (127.1.124.129). New cstate: current_term: 1 leader_uuid: "766511b6ae0544c3a8eabf2ed87c344b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } health_report { overall_health: HEALTHY } } }
I20260808 13:07:47.522567 27632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.517329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53466 (local address 127.1.124.129:42343)
0808 13:07:47.517517 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:47.517522 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:47.517631 (+ 109us) server_negotiation.cc:407] Connection header received
0808 13:07:47.517825 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:47.517828 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:47.517924 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:47.518096 (+ 172us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:07:47.518402 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.519188 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.520589 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.520996 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.521100 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:47.521563 (+ 463us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:07:47.522102 (+ 539us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:07:47.522215 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:47.522281 (+ 66us) server_negotiation.cc:299] Negotiation successful
0808 13:07:47.522353 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 13:07:47.531836 27543 raft_consensus.cc:1261] T 19f0115622d745a6b331dbd395e6d116 P 9db875a520d44bb693c809b4a6ea9537 [term 1 FOLLOWER]: Refusing update from remote peer 766511b6ae0544c3a8eabf2ed87c344b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:07:47.531836 27341 raft_consensus.cc:1261] T 19f0115622d745a6b331dbd395e6d116 P 0d0ea86bb8464914b0cd04ed16aba960 [term 1 FOLLOWER]: Refusing update from remote peer 766511b6ae0544c3a8eabf2ed87c344b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:07:47.533104 27650 consensus_queue.cc:1035] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:07:47.533494 27646 consensus_queue.cc:1035] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [LEADER]: Connected to new peer: Peer: permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:07:47.545953 27657 mvcc.cc:205] Tried to move back new op lower bound from 7316252538998681600 to 7316252538736087040. Current Snapshot: MvccSnapshot[applied={T|T < 7316252538998681600}]
I20260808 13:07:47.547216 27653 mvcc.cc:205] Tried to move back new op lower bound from 7316252538998681600 to 7316252538736087040. Current Snapshot: MvccSnapshot[applied={T|T < 7316252538998681600}]
I20260808 13:07:47.550019 27655 mvcc.cc:205] Tried to move back new op lower bound from 7316252538998681600 to 7316252538736087040. Current Snapshot: MvccSnapshot[applied={T|T < 7316252538998681600}]
I20260808 13:07:47.560191 26910 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:07:47.560489 26910 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43112:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:07:47.564349 26910 catalog_manager.cc:5782] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90: Sending DeleteTablet for 3 replicas of tablet 19f0115622d745a6b331dbd395e6d116
I20260808 13:07:47.565725 27518 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f0115622d745a6b331dbd395e6d116 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:07:47 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:40478
I20260808 13:07:47.565773 27314 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f0115622d745a6b331dbd395e6d116 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:07:47 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:40674
I20260808 13:07:47.565892 27106 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f0115622d745a6b331dbd395e6d116 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:07:47 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:53456
I20260808 13:07:47.567296 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 26970
I20260808 13:07:47.579114 27666 tablet_replica.cc:330] T 19f0115622d745a6b331dbd395e6d116 P 0d0ea86bb8464914b0cd04ed16aba960: stopping tablet replica
I20260808 13:07:47.579944 27671 tablet_replica.cc:330] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b: stopping tablet replica
I20260808 13:07:47.580765 27666 raft_consensus.cc:2229] T 19f0115622d745a6b331dbd395e6d116 P 0d0ea86bb8464914b0cd04ed16aba960 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:07:47.581478 27671 raft_consensus.cc:2229] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:07:47.582459 27671 raft_consensus.cc:2258] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:07:47.583202 27666 raft_consensus.cc:2258] T 19f0115622d745a6b331dbd395e6d116 P 0d0ea86bb8464914b0cd04ed16aba960 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:07:47.586522 27666 ts_tablet_manager.cc:1905] T 19f0115622d745a6b331dbd395e6d116 P 0d0ea86bb8464914b0cd04ed16aba960: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:07:47.589103 27667 tablet_replica.cc:330] T 19f0115622d745a6b331dbd395e6d116 P 9db875a520d44bb693c809b4a6ea9537: stopping tablet replica
I20260808 13:07:47.589110 27671 ts_tablet_manager.cc:1905] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:07:47.589560 27667 raft_consensus.cc:2229] T 19f0115622d745a6b331dbd395e6d116 P 9db875a520d44bb693c809b4a6ea9537 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:07:47.590104 27667 raft_consensus.cc:2258] T 19f0115622d745a6b331dbd395e6d116 P 9db875a520d44bb693c809b4a6ea9537 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:07:47.591436 27632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.568650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53476 (local address 127.1.124.129:42343)
0808 13:07:47.568882 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:47.568891 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:47.568916 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:07:47.568992 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:47.568998 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:47.569110 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:47.569244 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:47.571938 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.573757 (+ 1819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.577369 (+ 3612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.578393 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.578566 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:47.582916 (+ 4350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:47.582931 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:47.582950 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:47.582992 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:47.587322 (+ 4330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:47.588434 (+ 1112us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:47.588444 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:47.588448 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:47.588643 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:47.589183 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:47.589188 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:47.589192 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:47.589647 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:47.590007 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:47.590796 (+ 789us) server_negotiation.cc:299] Negotiation successful
0808 13:07:47.591050 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 13:07:47.592176 27204 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:07:47.593276 27667 ts_tablet_manager.cc:1905] T 19f0115622d745a6b331dbd395e6d116 P 9db875a520d44bb693c809b4a6ea9537: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:07:47.599395 27666 ts_tablet_manager.cc:1918] T 19f0115622d745a6b331dbd395e6d116 P 0d0ea86bb8464914b0cd04ed16aba960: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:07:47.599716 27666 log.cc:1196] T 19f0115622d745a6b331dbd395e6d116 P 0d0ea86bb8464914b0cd04ed16aba960: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/19f0115622d745a6b331dbd395e6d116
I20260808 13:07:47.600476 27666 ts_tablet_manager.cc:1939] T 19f0115622d745a6b331dbd395e6d116 P 0d0ea86bb8464914b0cd04ed16aba960: Deleting consensus metadata
I20260808 13:07:47.600502 27667 ts_tablet_manager.cc:1918] T 19f0115622d745a6b331dbd395e6d116 P 9db875a520d44bb693c809b4a6ea9537: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:07:47.600690 27667 log.cc:1196] T 19f0115622d745a6b331dbd395e6d116 P 9db875a520d44bb693c809b4a6ea9537: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/19f0115622d745a6b331dbd395e6d116
I20260808 13:07:47.601121 27667 ts_tablet_manager.cc:1939] T 19f0115622d745a6b331dbd395e6d116 P 9db875a520d44bb693c809b4a6ea9537: Deleting consensus metadata
I20260808 13:07:47.602705 26892 catalog_manager.cc:4841] TS 0d0ea86bb8464914b0cd04ed16aba960 (127.1.124.130:46503): tablet 19f0115622d745a6b331dbd395e6d116 (table test-table [id=4f1626dba6644d65a4453ffbf5666085]) successfully deleted
I20260808 13:07:47.602672 26891 catalog_manager.cc:4841] TS 9db875a520d44bb693c809b4a6ea9537 (127.1.124.131:34113): tablet 19f0115622d745a6b331dbd395e6d116 (table test-table [id=4f1626dba6644d65a4453ffbf5666085]) successfully deleted
I20260808 13:07:47.704607 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 26970
W20260808 13:07:47.732528 26892 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.124.129:42343 (error 108)
I20260808 13:07:47.732793 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 27215
I20260808 13:07:47.734670 27633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.734167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:42343 (local address 127.0.0.1:53484)
0808 13:07:47.734377 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:47.734475 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:42343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260808 13:07:47.734935 26892 catalog_manager.cc:4584] TS 766511b6ae0544c3a8eabf2ed87c344b (127.1.124.129:42343): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 19f0115622d745a6b331dbd395e6d116: Network error: Client connection negotiation failed: client connection to 127.1.124.129:42343: connect: Connection refused (error 111)
I20260808 13:07:47.747608 27629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.734255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40686 (local address 127.1.124.130:46503)
0808 13:07:47.734450 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:47.734455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:47.734471 (+ 16us) server_negotiation.cc:407] Connection header received
0808 13:07:47.734526 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:47.734530 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:47.734589 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:47.734668 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:47.736445 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.737216 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.738600 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.739029 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.739127 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:47.741900 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:47.741913 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:47.741928 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:47.741960 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:47.744943 (+ 2983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:47.745657 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:47.745664 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:47.745666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:47.745786 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:47.746212 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:47.746215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:47.746217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:47.746583 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:47.746794 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:47.747074 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 13:07:47.747250 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 13:07:47.748342 27406 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:07:47.788533 27633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.788028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:42343 (local address 127.0.0.1:53490)
0808 13:07:47.788285 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:47.788389 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:42343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103}
I20260808 13:07:47.789563 27633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.789216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:42343 (local address 127.0.0.1:53496)
0808 13:07:47.789362 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:47.789460 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:42343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 13:07:47.839460 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 27215
I20260808 13:07:47.850437 27633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.849760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:42343 (local address 127.0.0.1:53512)
0808 13:07:47.850054 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:47.850194 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:42343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260808 13:07:47.851672 27633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.851221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:42343 (local address 127.0.0.1:53516)
0808 13:07:47.851405 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:47.851510 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:42343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260808 13:07:47.866556 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 27416
I20260808 13:07:47.880566 27631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.867461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40490 (local address 127.1.124.131:34113)
0808 13:07:47.867637 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 13:07:47.867641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:07:47.867661 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:07:47.867721 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:07:47.867725 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:07:47.867786 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:07:47.867897 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:07:47.869420 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.870136 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.871465 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:07:47.871857 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:07:47.871950 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:07:47.874842 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:07:47.874858 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:07:47.874875 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:07:47.874911 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:07:47.877710 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:47.878495 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:47.878503 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:47.878506 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:47.878649 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:07:47.879008 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:07:47.879011 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:07:47.879014 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:07:47.879519 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:07:47.879741 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 13:07:47.880039 (+ 298us) server_negotiation.cc:299] Negotiation successful
0808 13:07:47.880283 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 13:07:47.881281 27607 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:07:47.933049 27633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.932437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:42343 (local address 127.0.0.1:53524)
0808 13:07:47.932695 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:47.932814 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:42343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260808 13:07:47.934252 27633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:47.933787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:42343 (local address 127.0.0.1:53540)
0808 13:07:47.934010 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:47.934100 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:42343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260808 13:07:47.971771 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 27416
I20260808 13:07:47.996629 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 26866
I20260808 13:07:47.997869 26956 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:07:48.114745 27633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:07:48.114082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:42343 (local address 127.0.0.1:53542)
0808 13:07:48.114344 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 13:07:48.114481 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:42343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260808 13:07:48.114830 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 26866
I20260808 13:08:08.149338 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:39149
--webserver_interface=127.1.124.190
--webserver_port=43803
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:45281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:39149
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:08:08.510166 27682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:08.510468 27682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:08.510623 27682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:08.518121 27682 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 13:08:08.518239 27682 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 13:08:08.518307 27682 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 13:08:08.518391 27682 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:08:08.518483 27682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:08.518620 27682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:08.518688 27682 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:08:08.518769 27682 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:08:08.528904 27682 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:45281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:39149
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:39149
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=43803
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:08.530267 27682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:08.531662 27682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:08.544654 27693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:08.544725 27694 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:08.544727 27696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:08.544982 27682 server_base.cc:1034] running on GCE node
I20260808 13:08:08.546154 27682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:08.547554 27682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:08.548810 27682 hybrid_clock.cc:648] HybridClock initialized: now 1786194488548751 us; error 32 us; skew 500 ppm
I20260808 13:08:08.549192 27682 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:08:08 dist-test-slave-t3q3 krb5kdc[26021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194488, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:08:08.552537 27682 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:08:08.554013 27682 webserver.cc:460] Webserver started at http://127.1.124.190:43803/ using document root <none> and password file <none>
I20260808 13:08:08.554670 27682 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:08.554764 27682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:08.560432 27682 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 13:08:08.563376 27706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:08.564895 27682 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:08:08.565089 27682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "da2f58b07de549af9773d27e893bdb90"
format_stamp: "Formatted at 2026-08-08 13:07:22 on dist-test-slave-t3q3"
server_key: "bb983c92fb988c4423aef9a11fd102af"
server_key_iv: "47173ca5213611f84bf756385a9d7a55"
server_key_version: "encryptionkey@0"
I20260808 13:08:08.566121 27682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:08.583578 27682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:08.588148 27682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:08.588502 27682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:08.608773 27682 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:39149
I20260808 13:08:08.608822 27790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:39149 every 8 connection(s)
I20260808 13:08:08.610350 27682 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:08:08.613837 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 27682
I20260808 13:08:08.614910 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:42343
--local_ip_for_outbound_sockets=127.1.124.129
--tserver_master_addrs=127.1.124.190:39149
--webserver_port=46633
--webserver_interface=127.1.124.129
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:45281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:08:08.619048 27791 sys_catalog.cc:263] Verifying existing consensus state
I20260808 13:08:08.624234 27791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90: Bootstrap starting.
I20260808 13:08:08.646001 27791 log.cc:824] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:08.662683 27791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 13:08:08.663283 27791 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90: Bootstrap complete.
I20260808 13:08:08.673004 27791 raft_consensus.cc:348] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2f58b07de549af9773d27e893bdb90" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39149 } }
I20260808 13:08:08.673988 27791 raft_consensus.cc:729] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: da2f58b07de549af9773d27e893bdb90, State: Initialized, Role: FOLLOWER
I20260808 13:08:08.674625 27791 consensus_queue.cc:260] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2f58b07de549af9773d27e893bdb90" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39149 } }
I20260808 13:08:08.674875 27791 raft_consensus.cc:388] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:08:08.674993 27791 raft_consensus.cc:482] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:08:08.675197 27791 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 2 FOLLOWER]: Advancing to term 3
I20260808 13:08:08.679198 27791 raft_consensus.cc:504] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2f58b07de549af9773d27e893bdb90" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39149 } }
I20260808 13:08:08.679698 27791 leader_election.cc:302] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da2f58b07de549af9773d27e893bdb90; no voters:
I20260808 13:08:08.680509 27791 leader_election.cc:288] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 13:08:08.680676 27796 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 13:08:08.681931 27796 raft_consensus.cc:686] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [term 3 LEADER]: Becoming Leader. State: Replica: da2f58b07de549af9773d27e893bdb90, State: Running, Role: LEADER
I20260808 13:08:08.682564 27796 consensus_queue.cc:237] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2f58b07de549af9773d27e893bdb90" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39149 } }
I20260808 13:08:08.682799 27791 sys_catalog.cc:564] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:08:08.685288 27798 sys_catalog.cc:455] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da2f58b07de549af9773d27e893bdb90. Latest consensus state: current_term: 3 leader_uuid: "da2f58b07de549af9773d27e893bdb90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2f58b07de549af9773d27e893bdb90" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39149 } } }
I20260808 13:08:08.685335 27797 sys_catalog.cc:455] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "da2f58b07de549af9773d27e893bdb90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da2f58b07de549af9773d27e893bdb90" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39149 } } }
I20260808 13:08:08.685715 27798 sys_catalog.cc:458] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [sys.catalog]: This master's current role is: LEADER
I20260808 13:08:08.685720 27797 sys_catalog.cc:458] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90 [sys.catalog]: This master's current role is: LEADER
I20260808 13:08:08.698748 27804 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:08:08.709204 27804 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:08:08.710806 27804 catalog_manager.cc:1259] Loaded cluster ID: 3d8168421f97430abc3d8d1aa248360a
I20260808 13:08:08.711006 27804 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:08:08.715634 27804 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:08:08.717691 27804 catalog_manager.cc:5879] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90: Loaded TSK: 0
I20260808 13:08:08.717936 27804 catalog_manager.cc:5879] T 00000000000000000000000000000000 P da2f58b07de549af9773d27e893bdb90: Loaded TSK: 1
F20260808 13:08:08.719108 27804 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fc8294569ad google::LogMessage::Fail() at ??:0
@ 0x7fc82945988a google::LogMessage::SendToLog() at ??:0
@ 0x7fc8294564de google::LogMessage::Flush() at ??:0
@ 0x7fc829457a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fc82d1ba7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fc82d1a973e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fc83ee851c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fc83ee84783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fc83ef102a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7fc83ef100c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7fc83ef5a5e9 std::function<>::operator()() at ??:0
@ 0x7fc83ee88cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7fc83ee86c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fc83ef0e610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7fc83ef0e439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc82c0a2345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fc82c0a591c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7fc82c0a5799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7fc82c080eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7fc82f35a6db start_thread at ??:0
@ 0x7fc8282f061f clone at ??:0
W20260808 13:08:09.079577 27794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:09.079883 27794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:09.079983 27794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:09.087410 27794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:09.087549 27794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:09.087738 27794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:08:09.097916 27794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:45281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:42343
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=46633
--tserver_master_addrs=127.1.124.190:39149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:09.099361 27794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:09.100785 27794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:09.115168 27866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:09.115226 27870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:09.115134 27865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:09.116698 27794 server_base.cc:1034] running on GCE node
I20260808 13:08:09.117317 27794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:09.118386 27794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:09.119683 27794 hybrid_clock.cc:648] HybridClock initialized: now 1786194489119638 us; error 59 us; skew 500 ppm
I20260808 13:08:09.120196 27794 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:08:09 dist-test-slave-t3q3 krb5kdc[26021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194489, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:08:09.123698 27794 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:08:09.125403 27794 webserver.cc:460] Webserver started at http://127.1.124.129:46633/ using document root <none> and password file <none>
I20260808 13:08:09.126112 27794 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:09.126258 27794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:09.132114 27794 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 13:08:09.135099 27878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:09.136461 27794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:08:09.136598 27794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "766511b6ae0544c3a8eabf2ed87c344b"
format_stamp: "Formatted at 2026-08-08 13:07:23 on dist-test-slave-t3q3"
server_key: "47193f2a5ab60fb8c51e9a26256d237b"
server_key_iv: "eac434ea418f52172c97240e134831c3"
server_key_version: "encryptionkey@0"
I20260808 13:08:09.137540 27794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:09.168581 27794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:09.172674 27794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:09.172906 27794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:09.173976 27794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:08:09.175752 27891 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260808 13:08:09.181478 27794 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 13:08:09.181581 27794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260808 13:08:09.181685 27794 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 13:08:09.184796 27794 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 13:08:09.184871 27794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260808 13:08:09.185206 27891 tablet_bootstrap.cc:492] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b: Bootstrap starting.
I20260808 13:08:09.193622 27891 log.cc:824] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:09.209847 27891 tablet_bootstrap.cc:492] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 13:08:09.210521 27891 tablet_bootstrap.cc:492] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b: Bootstrap complete.
I20260808 13:08:09.211062 27891 ts_tablet_manager.cc:1397] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b: Time spent bootstrapping tablet: real 0.026s user 0.021s sys 0.001s
I20260808 13:08:09.222743 27891 raft_consensus.cc:348] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:08:09.223800 27891 raft_consensus.cc:729] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 766511b6ae0544c3a8eabf2ed87c344b, State: Initialized, Role: FOLLOWER
I20260808 13:08:09.224769 27891 consensus_queue.cc:260] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:08:09.226507 27891 ts_tablet_manager.cc:1428] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260808 13:08:09.233935 27794 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:42343
I20260808 13:08:09.234014 28058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:42343 every 8 connection(s)
I20260808 13:08:09.235263 27794 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:08:09.239806 28063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:09.238021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.129:34009)
0808 13:08:09.238768 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:09.239001 (+ 233us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":315,"threads_started":1}
W20260808 13:08:09.240535 28059 heartbeater.cc:644] Failed to heartbeat to 127.1.124.190:39149 (0 consecutive failures): Network error: Failed to ping master at 127.1.124.190:39149: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
I20260808 13:08:09.241307 28063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:09.241017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.129:49359)
0808 13:08:09.241126 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:09.241209 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 13:08:09.241804 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 27794
I20260808 13:08:09.242311 28063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:09.241846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.129:50979)
0808 13:08:09.242041 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:09.242188 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260808 13:08:09.242522 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:46503
--local_ip_for_outbound_sockets=127.1.124.130
--tserver_master_addrs=127.1.124.190:39149
--webserver_port=34421
--webserver_interface=127.1.124.130
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:45281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:08:09.242542 28059 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 13:08:09.602561 28065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:09.602870 28065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:09.603029 28065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:09.610443 28065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:09.610569 28065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:09.610774 28065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:08:09.620920 28065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:45281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:46503
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=34421
--tserver_master_addrs=127.1.124.190:39149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:09.622247 28065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:09.623570 28065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:09.636821 28073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:09.636868 28078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:09.636778 28075 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:09.637799 28065 server_base.cc:1034] running on GCE node
I20260808 13:08:09.638407 28065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:09.639348 28065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:09.640580 28065 hybrid_clock.cc:648] HybridClock initialized: now 1786194489640522 us; error 54 us; skew 500 ppm
I20260808 13:08:09.640920 28065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:08:09 dist-test-slave-t3q3 krb5kdc[26021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194489, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:08:09.644385 28065 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:08:09.645994 28065 webserver.cc:460] Webserver started at http://127.1.124.130:34421/ using document root <none> and password file <none>
I20260808 13:08:09.646696 28065 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:09.646806 28065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:09.652488 28065 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:08:09.655232 28090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:09.656616 28065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:08:09.656802 28065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "0d0ea86bb8464914b0cd04ed16aba960"
format_stamp: "Formatted at 2026-08-08 13:07:23 on dist-test-slave-t3q3"
server_key: "ad2b3387acab3f88f20938211d1f38b1"
server_key_iv: "debb18a0e740dc23c6f066a4d24f60b8"
server_key_version: "encryptionkey@0"
I20260808 13:08:09.657789 28065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:09.686569 28065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:09.691481 28065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:09.691779 28065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:09.692920 28065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:08:09.694375 28065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:08:09.694471 28065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:09.694576 28065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:08:09.694654 28065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:09.731179 28065 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:46503
I20260808 13:08:09.731280 28257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:46503 every 8 connection(s)
I20260808 13:08:09.732690 28065 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:08:09.735694 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 28065
I20260808 13:08:09.736492 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:34113
--local_ip_for_outbound_sockets=127.1.124.131
--tserver_master_addrs=127.1.124.190:39149
--webserver_port=39963
--webserver_interface=127.1.124.131
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:45281
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:08:09.737812 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:09.735688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.130:34393)
0808 13:08:09.736616 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:09.736925 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":443,"threads_started":1}
W20260808 13:08:09.738580 28258 heartbeater.cc:644] Failed to heartbeat to 127.1.124.190:39149 (0 consecutive failures): Network error: Failed to ping master at 127.1.124.190:39149: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
I20260808 13:08:09.739441 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:09.739095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.130:53673)
0808 13:08:09.739255 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:09.739340 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 13:08:09.740585 28260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:09.740085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.130:40983)
0808 13:08:09.740315 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:09.740415 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260808 13:08:09.740856 28258 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 13:08:10.084363 28261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:10.084641 28261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:10.084738 28261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:10.091902 28261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:10.092017 28261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:10.092147 28261 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:08:10.101660 28261 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:45281
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:34113
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=39963
--tserver_master_addrs=127.1.124.190:39149
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:10.102905 28261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:10.104204 28261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:10.118315 28272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:10.118346 28269 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:10.118611 28267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:10.119261 28261 server_base.cc:1034] running on GCE node
I20260808 13:08:10.120033 28261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:10.121068 28261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:10.122313 28261 hybrid_clock.cc:648] HybridClock initialized: now 1786194490122267 us; error 61 us; skew 500 ppm
I20260808 13:08:10.122654 28261 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:08:10 dist-test-slave-t3q3 krb5kdc[26021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194490, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:08:10.125976 28261 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:08:10.127439 28261 webserver.cc:460] Webserver started at http://127.1.124.131:39963/ using document root <none> and password file <none>
I20260808 13:08:10.128196 28261 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:10.128348 28261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:10.133630 28261 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:08:10.136602 28282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:10.137995 28261 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:08:10.138178 28261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "9db875a520d44bb693c809b4a6ea9537"
format_stamp: "Formatted at 2026-08-08 13:07:24 on dist-test-slave-t3q3"
server_key: "11977007e82c74b72c46429bb4df8250"
server_key_iv: "25a660ee84e4f7154333d89d115916d3"
server_key_version: "encryptionkey@0"
I20260808 13:08:10.139186 28261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:10.154021 28261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:10.158470 28261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:10.158766 28261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:10.159718 28261 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:08:10.161381 28261 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:08:10.161517 28261 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:10.161643 28261 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:08:10.161737 28261 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:10.195442 28261 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:34113
I20260808 13:08:10.195537 28456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:34113 every 8 connection(s)
I20260808 13:08:10.197202 28261 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:08:10.198553 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 28261
I20260808 13:08:10.202945 28463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:10.201067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.131:33617)
0808 13:08:10.201792 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:10.202043 (+ 251us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":280,"threads_started":1}
W20260808 13:08:10.203775 28457 heartbeater.cc:644] Failed to heartbeat to 127.1.124.190:39149 (0 consecutive failures): Network error: Failed to ping master at 127.1.124.190:39149: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
I20260808 13:08:10.204594 28463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:10.204288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.131:43339)
0808 13:08:10.204423 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:10.204493 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 13:08:10.205466 28463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:10.205163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.131:51707)
0808 13:08:10.205282 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:10.205356 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 13:08:10.205735 28457 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 13:08:10.244144 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:10.243123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.129:37845)
0808 13:08:10.243653 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:10.243752 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":256,"threads_started":1}
I20260808 13:08:10.351555 28466 raft_consensus.cc:482] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:08:10.351891 28466 raft_consensus.cc:504] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:08:10.353683 28466 leader_election.cc:288] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d0ea86bb8464914b0cd04ed16aba960 (127.1.124.130:46503), 9db875a520d44bb693c809b4a6ea9537 (127.1.124.131:34113)
Aug 08 13:08:10 dist-test-slave-t3q3 krb5kdc[26021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194489, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.130@KRBTEST.COM
Aug 08 13:08:10 dist-test-slave-t3q3 krb5kdc[26021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194489, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.131@KRBTEST.COM
I20260808 13:08:10.379963 28465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:10.354066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:46503 (local address 127.1.124.129:51551)
0808 13:08:10.354443 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:10.354488 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:10.355214 (+ 726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:10.359121 (+ 3907us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:10.359127 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:10.359907 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:08:10.361006 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:10.361034 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:10.363494 (+ 2460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:10.363498 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:10.365986 (+ 2488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:10.366003 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:10.367532 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:10.367539 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:10.367951 (+ 412us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:10.368937 (+ 986us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:08:10.368977 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:08:10.372236 (+ 3259us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:08:10.376158 (+ 3922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:08:10.376171 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:08:10.376187 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:08:10.376786 (+ 599us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:08:10.377251 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:08:10.377256 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:08:10.377262 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:08:10.377455 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:08:10.378273 (+ 818us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:08:10.378284 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:08:10.378653 (+ 369us) client_negotiation.cc:769] Sending connection context
0808 13:08:10.378922 (+ 269us) client_negotiation.cc:240] Negotiation successful
0808 13:08:10.379501 (+ 579us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":355,"spinlock_wait_cycles":27264}
I20260808 13:08:10.379987 28468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:10.354326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:51551 (local address 127.1.124.130:46503)
0808 13:08:10.356770 (+ 2444us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:10.356782 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:10.356815 (+ 33us) server_negotiation.cc:407] Connection header received
0808 13:08:10.358311 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:10.358322 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:10.358739 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:10.359305 (+ 566us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:10.361235 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:10.363252 (+ 2017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:10.366164 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:10.367236 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:10.367431 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:10.372583 (+ 5152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:10.372596 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:10.372616 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:10.372678 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:10.375961 (+ 3283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:10.376950 (+ 989us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:10.376956 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:10.376963 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:10.377128 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:10.377584 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:10.377593 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:10.377595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:10.378116 (+ 521us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:10.378331 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:10.379169 (+ 838us) server_negotiation.cc:299] Negotiation successful
0808 13:08:10.379549 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2274,"thread_start_us":2167,"threads_started":1}
I20260808 13:08:10.380009 28470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:10.354428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:37895 (local address 127.1.124.131:34113)
0808 13:08:10.356467 (+ 2039us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:10.356481 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:10.356537 (+ 56us) server_negotiation.cc:407] Connection header received
0808 13:08:10.358005 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:10.358014 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:10.358388 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:10.358773 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:10.361235 (+ 2462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:10.363252 (+ 2017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:10.366178 (+ 2926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:10.367236 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:10.367436 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:10.372867 (+ 5431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:10.372875 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:10.372888 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:10.372927 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:10.376103 (+ 3176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:10.376968 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:10.376974 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:10.376980 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:10.377140 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:10.377613 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:10.377617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:10.377619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:10.378122 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:10.378333 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:10.379165 (+ 832us) server_negotiation.cc:299] Negotiation successful
0808 13:08:10.379528 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1862,"thread_start_us":1673,"threads_started":1}
I20260808 13:08:10.380091 28467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:10.354108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:34113 (local address 127.1.124.129:37895)
0808 13:08:10.356108 (+ 2000us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:10.356135 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:10.356258 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:10.358931 (+ 2673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:10.358945 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:10.360316 (+ 1371us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:08:10.361013 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:10.361034 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:10.363490 (+ 2456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:10.363496 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:10.365986 (+ 2490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:10.366014 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:10.367514 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:10.367522 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:10.367953 (+ 431us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:10.368941 (+ 988us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:08:10.368976 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:08:10.372592 (+ 3616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:08:10.376249 (+ 3657us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:08:10.376253 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:08:10.376256 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:08:10.376796 (+ 540us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:08:10.377281 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:08:10.377286 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:08:10.377290 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:08:10.377462 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:08:10.378277 (+ 815us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:08:10.378285 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:08:10.378649 (+ 364us) client_negotiation.cc:769] Sending connection context
0808 13:08:10.378923 (+ 274us) client_negotiation.cc:240] Negotiation successful
0808 13:08:10.379658 (+ 735us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1811,"mutex_wait_us":641,"spinlock_wait_cycles":40064,"thread_start_us":1627,"threads_started":1}
W20260808 13:08:10.384691 27882 leader_election.cc:341] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d0ea86bb8464914b0cd04ed16aba960 (127.1.124.130:46503): Not found: Tablet not found: 19f0115622d745a6b331dbd395e6d116
W20260808 13:08:10.385066 27881 leader_election.cc:341] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9db875a520d44bb693c809b4a6ea9537 (127.1.124.131:34113): Not found: Tablet not found: 19f0115622d745a6b331dbd395e6d116
I20260808 13:08:10.385219 27881 leader_election.cc:302] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 766511b6ae0544c3a8eabf2ed87c344b; no voters: 0d0ea86bb8464914b0cd04ed16aba960, 9db875a520d44bb693c809b4a6ea9537
I20260808 13:08:10.385598 28466 raft_consensus.cc:2726] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:08:10.742390 28471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:10.741380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.130:46709)
0808 13:08:10.741970 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:10.742148 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":323,"threads_started":1}
I20260808 13:08:11.207158 28473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:11.206358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.131:48019)
0808 13:08:11.206821 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:11.206939 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":234,"threads_started":1}
I20260808 13:08:11.245730 28475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:11.245091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.129:39627)
0808 13:08:11.245495 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:11.245573 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":208,"threads_started":1}
I20260808 13:08:11.744186 28476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:11.743430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.130:40937)
0808 13:08:11.743857 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:11.743940 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":228,"threads_started":1}
I20260808 13:08:12.209076 28477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:12.208208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.131:33493)
0808 13:08:12.208781 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:12.208872 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":311,"threads_started":1}
I20260808 13:08:12.247473 28478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:12.246659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.129:60573)
0808 13:08:12.247197 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:12.247288 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":281,"threads_started":1}
I20260808 13:08:12.331462 28479 raft_consensus.cc:482] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:08:12.331717 28479 raft_consensus.cc:504] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:08:12.332444 28479 leader_election.cc:288] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d0ea86bb8464914b0cd04ed16aba960 (127.1.124.130:46503), 9db875a520d44bb693c809b4a6ea9537 (127.1.124.131:34113)
W20260808 13:08:12.333469 27881 leader_election.cc:341] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9db875a520d44bb693c809b4a6ea9537 (127.1.124.131:34113): Not found: Tablet not found: 19f0115622d745a6b331dbd395e6d116
W20260808 13:08:12.333669 27882 leader_election.cc:341] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d0ea86bb8464914b0cd04ed16aba960 (127.1.124.130:46503): Not found: Tablet not found: 19f0115622d745a6b331dbd395e6d116
I20260808 13:08:12.333791 27882 leader_election.cc:302] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 766511b6ae0544c3a8eabf2ed87c344b; no voters: 0d0ea86bb8464914b0cd04ed16aba960, 9db875a520d44bb693c809b4a6ea9537
I20260808 13:08:12.334024 28479 raft_consensus.cc:2726] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:08:12.745764 28480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:12.745047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.130:39101)
0808 13:08:12.745514 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:12.745599 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":251,"threads_started":1}
I20260808 13:08:13.210698 28481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:13.209985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.131:44131)
0808 13:08:13.210432 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:13.210522 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":234,"threads_started":1}
I20260808 13:08:13.249444 28483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:13.248648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.129:58335)
0808 13:08:13.249187 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:13.249275 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":321,"threads_started":1}
I20260808 13:08:13.747355 28484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:13.746633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.130:58385)
0808 13:08:13.747088 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:13.747169 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":224,"threads_started":1}
I20260808 13:08:14.212601 28486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:14.211660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.131:35933)
0808 13:08:14.212294 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:14.212397 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":529,"thread_start_us":293,"threads_started":1}
I20260808 13:08:14.251206 28488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:14.250428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.129:48021)
0808 13:08:14.250905 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:14.250994 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":223,"threads_started":1}
I20260808 13:08:14.419575 28489 raft_consensus.cc:482] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:08:14.419850 28489 raft_consensus.cc:504] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:08:14.420615 28489 leader_election.cc:288] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d0ea86bb8464914b0cd04ed16aba960 (127.1.124.130:46503), 9db875a520d44bb693c809b4a6ea9537 (127.1.124.131:34113)
W20260808 13:08:14.421603 27882 leader_election.cc:341] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d0ea86bb8464914b0cd04ed16aba960 (127.1.124.130:46503): Not found: Tablet not found: 19f0115622d745a6b331dbd395e6d116
W20260808 13:08:14.421772 27881 leader_election.cc:341] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9db875a520d44bb693c809b4a6ea9537 (127.1.124.131:34113): Not found: Tablet not found: 19f0115622d745a6b331dbd395e6d116
I20260808 13:08:14.421866 27881 leader_election.cc:302] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 766511b6ae0544c3a8eabf2ed87c344b; no voters: 0d0ea86bb8464914b0cd04ed16aba960, 9db875a520d44bb693c809b4a6ea9537
I20260808 13:08:14.422081 28489 raft_consensus.cc:2726] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:08:14.749207 28495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:14.748442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.130:41679)
0808 13:08:14.748926 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:14.749014 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":260,"threads_started":1}
I20260808 13:08:15.214597 28498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:15.213718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.131:49617)
0808 13:08:15.214288 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:15.214431 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":331,"threads_started":1}
I20260808 13:08:15.252882 28499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:15.252203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.129:36007)
0808 13:08:15.252601 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:15.252707 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":201,"threads_started":1}
I20260808 13:08:15.751295 28501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:15.750241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.130:49419)
0808 13:08:15.750936 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:15.751054 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":396,"threads_started":1}
I20260808 13:08:16.216355 28502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:16.215550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.131:52127)
0808 13:08:16.216079 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:16.216172 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":276,"threads_started":1}
I20260808 13:08:16.254755 28503 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:16.253878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.129:50919)
0808 13:08:16.254373 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:16.254489 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":259,"threads_started":1}
I20260808 13:08:16.753265 28504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:16.752297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.130:60865)
0808 13:08:16.752955 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:16.753067 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":540,"thread_start_us":362,"threads_started":1}
I20260808 13:08:17.218307 28505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:17.217406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.131:39563)
0808 13:08:17.217980 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:17.218068 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":247,"threads_started":1}
I20260808 13:08:17.256438 28506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:17.255664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.129:46891)
0808 13:08:17.256194 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:17.256276 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":234,"threads_started":1}
I20260808 13:08:17.754997 28509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:17.754239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.130:37113)
0808 13:08:17.754703 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:17.754795 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":241,"threads_started":1}
I20260808 13:08:18.220247 28510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:18.219411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.131:50511)
0808 13:08:18.219889 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:18.219976 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":259,"threads_started":1}
I20260808 13:08:18.258149 28512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:18.257448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.129:60569)
0808 13:08:18.257872 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:18.257959 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":220,"threads_started":1}
I20260808 13:08:18.756788 28513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:18.755984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.130:51183)
0808 13:08:18.756530 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:18.756628 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":298,"threads_started":1}
I20260808 13:08:19.222119 28515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:19.221200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.131:47251)
0808 13:08:19.221825 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:19.221919 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":318,"threads_started":1}
I20260808 13:08:19.259948 28516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:19.259206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.129:55003)
0808 13:08:19.259695 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:19.259778 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":257,"threads_started":1}
I20260808 13:08:19.284859 28519 raft_consensus.cc:482] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:08:19.285156 28519 raft_consensus.cc:504] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:08:19.285892 28519 leader_election.cc:288] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d0ea86bb8464914b0cd04ed16aba960 (127.1.124.130:46503), 9db875a520d44bb693c809b4a6ea9537 (127.1.124.131:34113)
W20260808 13:08:19.287074 27882 leader_election.cc:341] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d0ea86bb8464914b0cd04ed16aba960 (127.1.124.130:46503): Not found: Tablet not found: 19f0115622d745a6b331dbd395e6d116
W20260808 13:08:19.287312 27881 leader_election.cc:341] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9db875a520d44bb693c809b4a6ea9537 (127.1.124.131:34113): Not found: Tablet not found: 19f0115622d745a6b331dbd395e6d116
I20260808 13:08:19.287478 27881 leader_election.cc:302] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 766511b6ae0544c3a8eabf2ed87c344b; no voters: 0d0ea86bb8464914b0cd04ed16aba960, 9db875a520d44bb693c809b4a6ea9537
I20260808 13:08:19.287768 28519 raft_consensus.cc:2726] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:08:19.758512 28520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:19.757816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.130:51295)
0808 13:08:19.758246 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:19.758329 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":228,"threads_started":1}
I20260808 13:08:20.224190 28524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:20.223316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.131:41841)
0808 13:08:20.223816 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:20.223908 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":261,"threads_started":1}
I20260808 13:08:20.261736 28525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:20.260864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.129:48155)
0808 13:08:20.261455 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:20.261545 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":367,"threads_started":1}
I20260808 13:08:20.760188 28526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:20.759402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.130:36709)
0808 13:08:20.759861 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:20.759944 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":226,"threads_started":1}
I20260808 13:08:21.225916 28527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:21.225158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.131:37009)
0808 13:08:21.225664 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:21.225752 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":248,"threads_started":1}
I20260808 13:08:21.263826 28529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:21.262787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.129:35187)
0808 13:08:21.263483 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:21.263602 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":392,"threads_started":1}
I20260808 13:08:21.762135 28532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:21.761247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.130:50353)
0808 13:08:21.761871 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:21.761960 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":395,"threads_started":1}
I20260808 13:08:22.227813 28533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:22.227066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.131:52293)
0808 13:08:22.227535 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:22.227643 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":253,"threads_started":1}
I20260808 13:08:22.265676 28534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:22.264746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.129:41169)
0808 13:08:22.265282 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:22.265391 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":296,"threads_started":1}
I20260808 13:08:22.763885 28537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:22.763111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.130:52531)
0808 13:08:22.763597 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:22.763703 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":242,"threads_started":1}
I20260808 13:08:23.061206 28538 raft_consensus.cc:482] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:08:23.061415 28538 raft_consensus.cc:504] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d0ea86bb8464914b0cd04ed16aba960" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 46503 } } peers { permanent_uuid: "9db875a520d44bb693c809b4a6ea9537" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 34113 } } peers { permanent_uuid: "766511b6ae0544c3a8eabf2ed87c344b" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42343 } }
I20260808 13:08:23.062180 28538 leader_election.cc:288] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0d0ea86bb8464914b0cd04ed16aba960 (127.1.124.130:46503), 9db875a520d44bb693c809b4a6ea9537 (127.1.124.131:34113)
W20260808 13:08:23.063112 27881 leader_election.cc:341] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 9db875a520d44bb693c809b4a6ea9537 (127.1.124.131:34113): Not found: Tablet not found: 19f0115622d745a6b331dbd395e6d116
W20260808 13:08:23.063297 27882 leader_election.cc:341] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0d0ea86bb8464914b0cd04ed16aba960 (127.1.124.130:46503): Not found: Tablet not found: 19f0115622d745a6b331dbd395e6d116
I20260808 13:08:23.063426 27882 leader_election.cc:302] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 766511b6ae0544c3a8eabf2ed87c344b; no voters: 0d0ea86bb8464914b0cd04ed16aba960, 9db875a520d44bb693c809b4a6ea9537
I20260808 13:08:23.063660 28538 raft_consensus.cc:2726] T 19f0115622d745a6b331dbd395e6d116 P 766511b6ae0544c3a8eabf2ed87c344b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 13:08:23.229846 28539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:23.228896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.131:58829)
0808 13:08:23.229552 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:23.229668 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":420,"threads_started":1}
I20260808 13:08:23.267813 28540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:23.266708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.129:41259)
0808 13:08:23.267335 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:23.267505 (+ 170us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":350,"threads_started":1}
I20260808 13:08:23.765865 28541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:23.764888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.130:51953)
0808 13:08:23.765460 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:23.765597 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":284,"threads_started":1}
I20260808 13:08:24.231727 28543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:24.230886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.131:51123)
0808 13:08:24.231428 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:24.231524 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":271,"threads_started":1}
I20260808 13:08:24.270020 28544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:24.269286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.129:37367)
0808 13:08:24.269757 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:24.269844 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":269,"threads_started":1}
I20260808 13:08:24.768373 28547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:24.767251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.130:52091)
0808 13:08:24.767872 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:24.768055 (+ 183us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":290,"threads_started":1}
I20260808 13:08:25.209394 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 27794
I20260808 13:08:25.225277 28548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:25.211022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54838 (local address 127.1.124.129:42343)
0808 13:08:25.211576 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:25.211589 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:25.211626 (+ 37us) server_negotiation.cc:407] Connection header received
0808 13:08:25.211711 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:25.211718 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:25.211946 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:25.212165 (+ 219us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:25.213629 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:25.214599 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:25.215945 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:25.216619 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:25.216730 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:25.219537 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:25.219548 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:25.219563 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:25.219605 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:25.222637 (+ 3032us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:25.223315 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:25.223323 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:25.223325 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:25.223456 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:25.223814 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:25.223817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:25.223819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:25.224283 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:25.224529 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:25.224785 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 13:08:25.224993 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":318,"threads_started":1}
I20260808 13:08:25.226550 28054 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:25.233819 28551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:25.232951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39149 (local address 127.1.124.131:56219)
0808 13:08:25.233521 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:25.233634 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.190:39149: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":267,"threads_started":1}
I20260808 13:08:25.322023 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 27794
I20260808 13:08:25.351753 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 28065
I20260808 13:08:25.367136 28554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:25.352621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45700 (local address 127.1.124.130:46503)
0808 13:08:25.353252 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:25.353257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:25.353274 (+ 17us) server_negotiation.cc:407] Connection header received
0808 13:08:25.353332 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:25.353336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:25.353402 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:25.353545 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:25.355135 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:25.355999 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:25.357452 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:25.357932 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:25.358147 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:25.361413 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:25.361419 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:25.361422 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:25.361459 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:25.364329 (+ 2870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:25.365033 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:25.365040 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:25.365044 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:25.365201 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:25.365609 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:25.365614 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:25.365619 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:25.365885 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:25.366178 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:25.366523 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 13:08:25.366749 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":283,"threads_started":1}
I20260808 13:08:25.367890 28252 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:25.453779 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 28065
I20260808 13:08:25.480254 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 28261
I20260808 13:08:25.494364 28556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:25.481176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50042 (local address 127.1.124.131:34113)
0808 13:08:25.481807 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:25.481813 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:25.481831 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:08:25.481890 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:25.481894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:25.481961 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:25.482056 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:25.483509 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:25.484430 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:25.485654 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:25.486085 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:25.486178 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:25.488911 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:25.488915 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:25.488919 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:25.488951 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:25.491825 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:25.492586 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:25.492590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:25.492592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:25.492724 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:25.493065 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:25.493068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:25.493070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:25.493348 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:25.493564 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:25.493776 (+ 212us) server_negotiation.cc:299] Negotiation successful
0808 13:08:25.493948 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":338,"threads_started":1}
I20260808 13:08:25.495081 28451 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:25.580608 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 28261
2026-08-08T13:08:25Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65251 ms)
[----------] 30 tests from SecurityITest (319885 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:08:25 dist-test-slave-t3q3 krb5kdc[28562](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:08:25 dist-test-slave-t3q3 krb5kdc[28562](info): set up 2 sockets
Aug 08 13:08:25 dist-test-slave-t3q3 krb5kdc[28562](info): commencing operation
krb5kdc: starting...
W20260808 13:08:26.941627 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:08:27 dist-test-slave-t3q3 krb5kdc[28562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:08:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:08:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:08:27.272689 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:37287
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:37715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:37287
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:08:27.651891 28581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:27.652281 28581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:27.652390 28581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:27.659816 28581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:08:27.659926 28581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:27.659974 28581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:27.660034 28581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:08:27.660144 28581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:08:27.670145 28581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:37715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:37287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:37287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:27.671453 28581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:27.672808 28581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:27.685096 28587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:27.685690 28590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:27.685748 28588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:27.686290 28581 server_base.cc:1034] running on GCE node
I20260808 13:08:27.687228 28581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:27.688824 28581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:27.690089 28581 hybrid_clock.cc:648] HybridClock initialized: now 1786194507690035 us; error 64 us; skew 500 ppm
I20260808 13:08:27.690457 28581 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:08:27 dist-test-slave-t3q3 krb5kdc[28562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194507, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:08:27.693745 28581 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:08:27.695497 28581 webserver.cc:460] Webserver started at http://127.1.124.190:34489/ using document root <none> and password file <none>
I20260808 13:08:27.696312 28581 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:27.696465 28581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:27.696792 28581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:27.699934 28581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "7559d68a45394071995a4f780503bc9f"
format_stamp: "Formatted at 2026-08-08 13:08:27 on dist-test-slave-t3q3"
server_key: "04bb2174ea2814dae3670413611a913b"
server_key_iv: "d29b662df613a29ec7e42c3e9e940fe9"
server_key_version: "encryptionkey@0"
I20260808 13:08:27.700839 28581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "7559d68a45394071995a4f780503bc9f"
format_stamp: "Formatted at 2026-08-08 13:08:27 on dist-test-slave-t3q3"
server_key: "04bb2174ea2814dae3670413611a913b"
server_key_iv: "d29b662df613a29ec7e42c3e9e940fe9"
server_key_version: "encryptionkey@0"
I20260808 13:08:27.706537 28581 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:08:27.710388 28602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:27.711855 28581 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:08:27.712090 28581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "7559d68a45394071995a4f780503bc9f"
format_stamp: "Formatted at 2026-08-08 13:08:27 on dist-test-slave-t3q3"
server_key: "04bb2174ea2814dae3670413611a913b"
server_key_iv: "d29b662df613a29ec7e42c3e9e940fe9"
server_key_version: "encryptionkey@0"
I20260808 13:08:27.712363 28581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:27.734424 28581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:27.739817 28581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:27.740288 28581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:27.761334 28581 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:37287
I20260808 13:08:27.761391 28683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:37287 every 8 connection(s)
I20260808 13:08:27.762871 28581 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:08:27.767192 28684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:27.767400 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 28581
I20260808 13:08:27.767647 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:08:27.768656 1522 external_mini_cluster.cc:1428] Setting key 2e910b5ec0023ef0c94d2e394b30bb11
I20260808 13:08:27.782642 28684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f: Bootstrap starting.
Aug 08 13:08:27 dist-test-slave-t3q3 krb5kdc[28562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:08:27.787492 28684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:27.788738 28684 log.cc:824] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:27.791927 28684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f: No bootstrap required, opened a new log
I20260808 13:08:27.795863 28690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:27.771023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47884 (local address 127.1.124.190:37287)
0808 13:08:27.771931 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:27.771946 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:27.771994 (+ 48us) server_negotiation.cc:407] Connection header received
0808 13:08:27.774138 (+ 2144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:27.774169 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:27.774607 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:27.774978 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:27.777076 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:27.778520 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:27.780531 (+ 2011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:27.781262 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:27.781416 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:27.787451 (+ 6035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:27.787461 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:27.787482 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:27.787528 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:27.791028 (+ 3500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:27.791920 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:27.791928 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:27.791935 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:27.792207 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:27.792660 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:27.792672 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:27.792674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:27.793285 (+ 611us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:27.793644 (+ 359us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:27.794205 (+ 561us) server_negotiation.cc:299] Negotiation successful
0808 13:08:27.794712 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":678,"thread_start_us":427,"threads_started":1}
I20260808 13:08:27.800786 28684 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7559d68a45394071995a4f780503bc9f" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37287 } }
I20260808 13:08:27.801221 28684 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:27.801349 28684 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7559d68a45394071995a4f780503bc9f, State: Initialized, Role: FOLLOWER
I20260808 13:08:27.802103 28684 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7559d68a45394071995a4f780503bc9f" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37287 } }
I20260808 13:08:27.802460 28684 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:08:27.802634 28684 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:08:27.802827 28684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:27.805752 28684 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7559d68a45394071995a4f780503bc9f" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37287 } }
I20260808 13:08:27.806406 28684 leader_election.cc:302] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7559d68a45394071995a4f780503bc9f; no voters:
I20260808 13:08:27.807068 28684 leader_election.cc:288] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:08:27.807252 28694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:08:27.808853 28694 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f [term 1 LEADER]: Becoming Leader. State: Replica: 7559d68a45394071995a4f780503bc9f, State: Running, Role: LEADER
I20260808 13:08:27.809537 28684 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:08:27.809445 28694 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7559d68a45394071995a4f780503bc9f" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37287 } }
I20260808 13:08:27.812597 28695 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7559d68a45394071995a4f780503bc9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7559d68a45394071995a4f780503bc9f" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37287 } } }
I20260808 13:08:27.813019 28695 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f [sys.catalog]: This master's current role is: LEADER
I20260808 13:08:27.813496 28696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7559d68a45394071995a4f780503bc9f. Latest consensus state: current_term: 1 leader_uuid: "7559d68a45394071995a4f780503bc9f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7559d68a45394071995a4f780503bc9f" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37287 } } }
I20260808 13:08:27.813831 28696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f [sys.catalog]: This master's current role is: LEADER
I20260808 13:08:27.815475 28707 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:08:27.821538 28707 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:08:27.831877 28707 catalog_manager.cc:1347] Generated new cluster ID: 11f7fec5f03f4bdb9c4477070286f6f3
I20260808 13:08:27.832054 28707 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:08:27.850353 28707 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:08:27.852010 28707 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:08:27.865993 28707 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f: Generated new TSK 0
I20260808 13:08:27.866971 28707 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.124.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
I20260808 13:08:27.950114 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:37287
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:37715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:08:28.304188 28723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:28.304502 28723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:28.304603 28723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:28.312049 28723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:28.312229 28723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:28.312347 28723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:08:28.322500 28723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:37715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:37287
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:28.323729 28723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:28.325079 28723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:28.338706 28730 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:28.338750 28732 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:28.338776 28729 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:28.339593 28723 server_base.cc:1034] running on GCE node
I20260808 13:08:28.340374 28723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:28.341380 28723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:28.342614 28723 hybrid_clock.cc:648] HybridClock initialized: now 1786194508342566 us; error 68 us; skew 500 ppm
I20260808 13:08:28.342954 28723 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:08:28 dist-test-slave-t3q3 krb5kdc[28562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194508, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:08:28.346362 28723 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:08:28.347865 28723 webserver.cc:460] Webserver started at http://127.1.124.129:40903/ using document root <none> and password file <none>
I20260808 13:08:28.348651 28723 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:28.348811 28723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:28.349155 28723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:28.352283 28723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "6255d2c9742a4a71904ae5cb718a82af"
format_stamp: "Formatted at 2026-08-08 13:08:28 on dist-test-slave-t3q3"
server_key: "fe6b1f9820ca17af9b2839929281899a"
server_key_iv: "fc5c5e569861ee7140e096d0cfaa9f1c"
server_key_version: "encryptionkey@0"
I20260808 13:08:28.353137 28723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "6255d2c9742a4a71904ae5cb718a82af"
format_stamp: "Formatted at 2026-08-08 13:08:28 on dist-test-slave-t3q3"
server_key: "fe6b1f9820ca17af9b2839929281899a"
server_key_iv: "fc5c5e569861ee7140e096d0cfaa9f1c"
server_key_version: "encryptionkey@0"
I20260808 13:08:28.358496 28723 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:08:28.362128 28746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:28.363534 28723 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:08:28.363725 28723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "6255d2c9742a4a71904ae5cb718a82af"
format_stamp: "Formatted at 2026-08-08 13:08:28 on dist-test-slave-t3q3"
server_key: "fe6b1f9820ca17af9b2839929281899a"
server_key_iv: "fc5c5e569861ee7140e096d0cfaa9f1c"
server_key_version: "encryptionkey@0"
I20260808 13:08:28.363945 28723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:28.385943 28723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:28.390565 28723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:28.390887 28723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:28.391917 28723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:08:28.393430 28723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:08:28.393543 28723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:28.393700 28723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:08:28.393778 28723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:28.428710 28723 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:34283
I20260808 13:08:28.428802 28911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:34283 every 8 connection(s)
I20260808 13:08:28.430032 28723 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:08:28.434113 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 28723
I20260808 13:08:28.434358 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:08:28.435050 1522 external_mini_cluster.cc:1428] Setting key d44135b20ae03d85b10213b8b8aba3b0
Aug 08 13:08:28 dist-test-slave-t3q3 krb5kdc[28562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194508, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:08:28.450562 28915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:28.432640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:38095 (local address 127.1.124.190:37287)
0808 13:08:28.433128 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:28.433133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:28.433970 (+ 837us) server_negotiation.cc:407] Connection header received
0808 13:08:28.436175 (+ 2205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:28.436181 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:28.436271 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:28.436392 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:28.438055 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:28.438783 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:28.440422 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:28.440840 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:28.440978 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:28.444608 (+ 3630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:28.444615 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:28.444619 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:28.444652 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:28.447346 (+ 2694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:28.448150 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:28.448154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:28.448157 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:28.448280 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:28.448745 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:28.448749 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:28.448751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:28.449015 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:28.449181 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:28.449925 (+ 744us) server_negotiation.cc:299] Negotiation successful
0808 13:08:28.450174 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":243,"threads_started":1}
I20260808 13:08:28.451545 28916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:28.433084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:37287 (local address 127.1.124.129:38095)
0808 13:08:28.433806 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:28.433844 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:28.435888 (+ 2044us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:28.436641 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:28.436649 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:28.437308 (+ 659us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:08:28.437879 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:28.437896 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:28.438953 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:28.438957 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:28.440278 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:28.440291 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:28.441077 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:28.441085 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:28.441326 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:28.442085 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:08:28.442108 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:08:28.444364 (+ 2256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:08:28.447502 (+ 3138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:08:28.447512 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:08:28.447529 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:08:28.447982 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:08:28.448397 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:08:28.448402 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:08:28.448406 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:08:28.448604 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:08:28.449162 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:08:28.449170 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:08:28.449525 (+ 355us) client_negotiation.cc:769] Sending connection context
0808 13:08:28.449808 (+ 283us) client_negotiation.cc:240] Negotiation successful
0808 13:08:28.450363 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":272,"threads_started":1}
I20260808 13:08:28.453243 28912 heartbeater.cc:347] Connected to a master server at 127.1.124.190:37287
I20260808 13:08:28.453549 28912 heartbeater.cc:464] Registering TS with master...
I20260808 13:08:28.454279 28912 heartbeater.cc:511] Master 127.1.124.190:37287 requested a full tablet report, sending...
I20260808 13:08:28.456645 28627 ts_manager.cc:194] Registered new tserver with Master: 6255d2c9742a4a71904ae5cb718a82af (127.1.124.129:34283)
I20260808 13:08:28.458940 28627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:38095
WARNING: no policy specified for kudu/127.1.124.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
I20260808 13:08:28.503592 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:37287
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:37715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:08:28.854996 28923 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:28.855289 28923 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:28.855384 28923 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:28.862737 28923 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:28.862857 28923 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:28.862970 28923 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:08:28.872849 28923 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:37715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:37287
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:28.874147 28923 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:28.875468 28923 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:28.889631 28930 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:28.889662 28932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:28.889716 28934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:28.890236 28923 server_base.cc:1034] running on GCE node
I20260808 13:08:28.890931 28923 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:28.891902 28923 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:28.893139 28923 hybrid_clock.cc:648] HybridClock initialized: now 1786194508893082 us; error 65 us; skew 500 ppm
I20260808 13:08:28.893468 28923 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:08:28 dist-test-slave-t3q3 krb5kdc[28562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194508, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:08:28.896838 28923 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:08:28.898370 28923 webserver.cc:460] Webserver started at http://127.1.124.130:46841/ using document root <none> and password file <none>
I20260808 13:08:28.899091 28923 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:28.899256 28923 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:28.899593 28923 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:28.902685 28923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "f2d4d9a2228f405d9df8aeef8cf6d17e"
format_stamp: "Formatted at 2026-08-08 13:08:28 on dist-test-slave-t3q3"
server_key: "b1d936d031680a2547e734180ff57fc1"
server_key_iv: "59ff4e16cef1fc8d71780cda17538b07"
server_key_version: "encryptionkey@0"
I20260808 13:08:28.903541 28923 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "f2d4d9a2228f405d9df8aeef8cf6d17e"
format_stamp: "Formatted at 2026-08-08 13:08:28 on dist-test-slave-t3q3"
server_key: "b1d936d031680a2547e734180ff57fc1"
server_key_iv: "59ff4e16cef1fc8d71780cda17538b07"
server_key_version: "encryptionkey@0"
I20260808 13:08:28.908998 28923 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260808 13:08:28.912497 28944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:28.913836 28923 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 13:08:28.914027 28923 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "f2d4d9a2228f405d9df8aeef8cf6d17e"
format_stamp: "Formatted at 2026-08-08 13:08:28 on dist-test-slave-t3q3"
server_key: "b1d936d031680a2547e734180ff57fc1"
server_key_iv: "59ff4e16cef1fc8d71780cda17538b07"
server_key_version: "encryptionkey@0"
I20260808 13:08:28.914248 28923 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:28.947839 28923 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:28.952155 28923 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:28.952469 28923 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:28.953490 28923 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:08:28.954895 28923 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:08:28.955008 28923 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:28.955175 28923 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:08:28.955260 28923 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:28.990437 28923 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:42511
I20260808 13:08:28.990492 29115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:42511 every 8 connection(s)
I20260808 13:08:28.991844 28923 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:08:28.995139 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 28923
I20260808 13:08:28.995347 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:08:28.995990 1522 external_mini_cluster.cc:1428] Setting key 9bf31cfa1b42200f6dcd1e3225df55eb
Aug 08 13:08:29 dist-test-slave-t3q3 krb5kdc[28562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194508, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:08:29.013861 29118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:28.994631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:52613 (local address 127.1.124.190:37287)
0808 13:08:28.995193 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:28.995199 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:28.996406 (+ 1207us) server_negotiation.cc:407] Connection header received
0808 13:08:28.998883 (+ 2477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:28.998888 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:28.998979 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:28.999102 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:29.000783 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.001579 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.003177 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.003623 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.003765 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:29.007456 (+ 3691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:29.007462 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:29.007466 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:29.007498 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:29.010363 (+ 2865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:29.011255 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:29.011262 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:29.011265 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:29.011440 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:29.011990 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:29.011994 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:29.011996 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:29.012347 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:29.012466 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:29.013406 (+ 940us) server_negotiation.cc:299] Negotiation successful
0808 13:08:29.013582 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":275,"threads_started":1}
I20260808 13:08:29.015498 29120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:28.995352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:37287 (local address 127.1.124.130:52613)
0808 13:08:28.996220 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:28.996276 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:28.998592 (+ 2316us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:28.999369 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:28.999378 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:28.999924 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:08:29.000601 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:29.000619 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.001738 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.001743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:29.003014 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:29.003027 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.003863 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.003870 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:29.004186 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:29.004905 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:08:29.004934 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:08:29.007216 (+ 2282us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:08:29.010511 (+ 3295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:08:29.010540 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:08:29.010558 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:08:29.011085 (+ 527us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:08:29.011577 (+ 492us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:08:29.011583 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:08:29.011588 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:08:29.011854 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:08:29.012482 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:08:29.012493 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:08:29.012978 (+ 485us) client_negotiation.cc:769] Sending connection context
0808 13:08:29.013320 (+ 342us) client_negotiation.cc:240] Negotiation successful
0808 13:08:29.013921 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":336,"threads_started":1}
I20260808 13:08:29.017184 29116 heartbeater.cc:347] Connected to a master server at 127.1.124.190:37287
I20260808 13:08:29.017547 29116 heartbeater.cc:464] Registering TS with master...
I20260808 13:08:29.018402 29116 heartbeater.cc:511] Master 127.1.124.190:37287 requested a full tablet report, sending...
I20260808 13:08:29.019979 28627 ts_manager.cc:194] Registered new tserver with Master: f2d4d9a2228f405d9df8aeef8cf6d17e (127.1.124.130:42511)
I20260808 13:08:29.021350 28627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:52613
WARNING: no policy specified for kudu/127.1.124.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
I20260808 13:08:29.068989 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:37287
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:37715
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:08:29.427634 29125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:29.428037 29125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:29.428228 29125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:29.438135 29125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:29.438243 29125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:29.438359 29125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:08:29.448021 29125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:37715
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:37287
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:29.449411 29125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:29.450809 29125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 13:08:29.462128 28912 heartbeater.cc:503] Master 127.1.124.190:37287 was elected leader, sending a full tablet report...
W20260808 13:08:29.465310 29131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:29.465193 29133 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:29.465463 29135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:29.466398 29125 server_base.cc:1034] running on GCE node
I20260808 13:08:29.467048 29125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:29.468256 29125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:29.469497 29125 hybrid_clock.cc:648] HybridClock initialized: now 1786194509469458 us; error 48 us; skew 500 ppm
I20260808 13:08:29.469861 29125 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:08:29 dist-test-slave-t3q3 krb5kdc[28562](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194509, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:08:29.473197 29125 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:08:29.474691 29125 webserver.cc:460] Webserver started at http://127.1.124.131:35011/ using document root <none> and password file <none>
I20260808 13:08:29.475368 29125 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:29.475524 29125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:29.475857 29125 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:29.478889 29125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "04f8c5c6225b4df1a88e9763b330fa9a"
format_stamp: "Formatted at 2026-08-08 13:08:29 on dist-test-slave-t3q3"
server_key: "39dd4e7761b9151626f0f0eca36bde70"
server_key_iv: "0e94769543bb6bb46a96106084f33e29"
server_key_version: "encryptionkey@0"
I20260808 13:08:29.479705 29125 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "04f8c5c6225b4df1a88e9763b330fa9a"
format_stamp: "Formatted at 2026-08-08 13:08:29 on dist-test-slave-t3q3"
server_key: "39dd4e7761b9151626f0f0eca36bde70"
server_key_iv: "0e94769543bb6bb46a96106084f33e29"
server_key_version: "encryptionkey@0"
I20260808 13:08:29.484874 29125 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260808 13:08:29.488366 29154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:29.489703 29125 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:08:29.489884 29125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "04f8c5c6225b4df1a88e9763b330fa9a"
format_stamp: "Formatted at 2026-08-08 13:08:29 on dist-test-slave-t3q3"
server_key: "39dd4e7761b9151626f0f0eca36bde70"
server_key_iv: "0e94769543bb6bb46a96106084f33e29"
server_key_version: "encryptionkey@0"
I20260808 13:08:29.490092 29125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:29.511895 29125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:29.516600 29125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:29.516929 29125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:29.518255 29125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:08:29.519901 29125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:08:29.520022 29125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:29.520231 29125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:08:29.520308 29125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:29.556696 29125 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:36015
I20260808 13:08:29.556758 29322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:36015 every 8 connection(s)
I20260808 13:08:29.558066 29125 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:08:29.560467 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 29125
I20260808 13:08:29.560700 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:08:29.561329 1522 external_mini_cluster.cc:1428] Setting key 13f7645d4b933f3c0cdadac68941f45a
Aug 08 13:08:29 dist-test-slave-t3q3 krb5kdc[28562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194509, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:08:29.579326 29327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:29.560835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:51661 (local address 127.1.124.190:37287)
0808 13:08:29.561347 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:29.561353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:29.562525 (+ 1172us) server_negotiation.cc:407] Connection header received
0808 13:08:29.564610 (+ 2085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:29.564615 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:29.564716 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:29.564808 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:29.566277 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.567047 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.568768 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.569251 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.569350 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:29.573065 (+ 3715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:29.573070 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:29.573074 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:29.573106 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:29.576259 (+ 3153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:29.577224 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:29.577230 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:29.577233 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:29.577368 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:29.577782 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:29.577785 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:29.577787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:29.578064 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:29.578205 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:29.578858 (+ 653us) server_negotiation.cc:299] Negotiation successful
0808 13:08:29.579022 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":246,"threads_started":1}
I20260808 13:08:29.580361 29328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:29.561477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:37287 (local address 127.1.124.131:51661)
0808 13:08:29.562349 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:29.562403 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:29.564292 (+ 1889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:29.565020 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:29.565027 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:29.565542 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:08:29.566127 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:29.566143 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.567172 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.567176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:29.568638 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:29.568651 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.569440 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.569444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:29.569654 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:29.570318 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:08:29.570340 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:08:29.572850 (+ 2510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:08:29.576407 (+ 3557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:08:29.576414 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:08:29.576427 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:08:29.577080 (+ 653us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:08:29.577464 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:08:29.577467 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:08:29.577469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:08:29.577669 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:08:29.578179 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:08:29.578185 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:08:29.578503 (+ 318us) client_negotiation.cc:769] Sending connection context
0808 13:08:29.578754 (+ 251us) client_negotiation.cc:240] Negotiation successful
0808 13:08:29.579142 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":598,"thread_start_us":382,"threads_started":1}
I20260808 13:08:29.581692 29323 heartbeater.cc:347] Connected to a master server at 127.1.124.190:37287
I20260808 13:08:29.582020 29323 heartbeater.cc:464] Registering TS with master...
I20260808 13:08:29.582762 29323 heartbeater.cc:511] Master 127.1.124.190:37287 requested a full tablet report, sending...
I20260808 13:08:29.584295 28627 ts_manager.cc:194] Registered new tserver with Master: 04f8c5c6225b4df1a88e9763b330fa9a (127.1.124.131:36015)
I20260808 13:08:29.585081 28627 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:51661
I20260808 13:08:29.589385 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:08:29.608417 29327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:29.594147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44062 (local address 127.1.124.190:37287)
0808 13:08:29.594348 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:29.594354 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:29.594847 (+ 493us) server_negotiation.cc:407] Connection header received
0808 13:08:29.595024 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:29.595027 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:29.595088 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:29.595188 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:29.596715 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.597449 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.598862 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.599276 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.599383 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:29.601729 (+ 2346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:29.601737 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:29.601741 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:29.601771 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:29.604794 (+ 3023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:29.605707 (+ 913us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:29.605714 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:29.605718 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:29.605890 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:29.606428 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:29.606435 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:29.606438 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:29.606965 (+ 527us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:29.607258 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:29.607716 (+ 458us) server_negotiation.cc:299] Negotiation successful
0808 13:08:29.607980 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 13:08:29.615301 28627 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44062:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:08:29.623092 28627 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:08:29.646543 29345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:29.634876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:36015 (local address 127.0.0.1:56002)
0808 13:08:29.637057 (+ 2181us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:29.637106 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:29.637335 (+ 229us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:29.638380 (+ 1045us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:29.638387 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:29.638419 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:29.638864 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:29.638877 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.640944 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.640949 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:29.643247 (+ 2298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:29.643263 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.645537 (+ 2274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.645543 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:29.645758 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:29.645823 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 13:08:29.645945 (+ 122us) client_negotiation.cc:240] Negotiation successful
0808 13:08:29.646133 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1854,"thread_start_us":1702,"threads_started":1}
I20260808 13:08:29.647468 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:29.635403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56002 (local address 127.1.124.131:36015)
0808 13:08:29.636662 (+ 1259us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:29.636672 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:29.637254 (+ 582us) server_negotiation.cc:407] Connection header received
0808 13:08:29.637606 (+ 352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:29.637615 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:29.637881 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:29.638069 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:29.639085 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.640779 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.643433 (+ 2654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.645017 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.645145 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:29.645367 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:29.646942 (+ 1575us) server_negotiation.cc:299] Negotiation successful
0808 13:08:29.647121 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1057,"thread_start_us":850,"threads_started":1}
I20260808 13:08:29.649753 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:29.635414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39316 (local address 127.1.124.129:34283)
0808 13:08:29.637177 (+ 1763us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:29.637185 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:29.637819 (+ 634us) server_negotiation.cc:407] Connection header received
0808 13:08:29.638165 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:29.638175 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:29.638388 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:29.638559 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:29.640626 (+ 2067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.642222 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.645719 (+ 3497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.647211 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.647323 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:29.647549 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:29.649237 (+ 1688us) server_negotiation.cc:299] Negotiation successful
0808 13:08:29.649420 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1606,"thread_start_us":1768,"threads_started":1}
I20260808 13:08:29.650470 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:29.635261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:34283 (local address 127.0.0.1:39316)
0808 13:08:29.637709 (+ 2448us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:29.637725 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:29.637931 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:29.639949 (+ 2018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:29.639954 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:29.639977 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:29.640420 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:29.640433 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.643367 (+ 2934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.643373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:29.645538 (+ 2165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:29.645551 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.648814 (+ 3263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.648820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:29.648991 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:29.649036 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 13:08:29.649953 (+ 917us) client_negotiation.cc:240] Negotiation successful
0808 13:08:29.650109 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2330,"thread_start_us":2155,"threads_started":1}
I20260808 13:08:29.652468 29222 tablet_service.cc:1467] Processing CreateTablet for tablet 6555716b39404eb39c01555040fb4c5d (DEFAULT_TABLE table=test-table [id=2aa25583f347446dbb108bf5858fa86d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:08:29.654179 29222 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6555716b39404eb39c01555040fb4c5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:29.654718 28815 tablet_service.cc:1467] Processing CreateTablet for tablet 6555716b39404eb39c01555040fb4c5d (DEFAULT_TABLE table=test-table [id=2aa25583f347446dbb108bf5858fa86d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:08:29.655921 28815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6555716b39404eb39c01555040fb4c5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:29.659469 29345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:29.637782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:42511 (local address 127.0.0.1:50098)
0808 13:08:29.648185 (+ 10403us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:29.648203 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:29.648291 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:29.651864 (+ 3573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:29.651868 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:29.651900 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:29.652117 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:29.652129 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.654075 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.654080 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:29.655862 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:29.655877 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.658653 (+ 2776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.658664 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:29.658856 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:29.658900 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 13:08:29.659014 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 13:08:29.659173 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10314,"thread_start_us":13307,"threads_started":1}
I20260808 13:08:29.660413 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:29.638114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50098 (local address 127.1.124.130:42511)
0808 13:08:29.651214 (+ 13100us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:29.651222 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:29.651257 (+ 35us) server_negotiation.cc:407] Connection header received
0808 13:08:29.651365 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:29.651371 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:29.651618 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:29.651844 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:29.652254 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.653903 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.656103 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.657584 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.657758 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:29.657964 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:29.659821 (+ 1857us) server_negotiation.cc:299] Negotiation successful
0808 13:08:29.660017 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12944,"thread_start_us":12817,"threads_started":1}
I20260808 13:08:29.664587 29015 tablet_service.cc:1467] Processing CreateTablet for tablet 6555716b39404eb39c01555040fb4c5d (DEFAULT_TABLE table=test-table [id=2aa25583f347446dbb108bf5858fa86d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:08:29.666111 29015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6555716b39404eb39c01555040fb4c5d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:29.668686 29358 tablet_bootstrap.cc:492] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a: Bootstrap starting.
I20260808 13:08:29.672832 29359 tablet_bootstrap.cc:492] T 6555716b39404eb39c01555040fb4c5d P 6255d2c9742a4a71904ae5cb718a82af: Bootstrap starting.
I20260808 13:08:29.673452 29358 tablet_bootstrap.cc:654] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:29.674773 29358 log.cc:824] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:29.676594 29359 tablet_bootstrap.cc:654] T 6555716b39404eb39c01555040fb4c5d P 6255d2c9742a4a71904ae5cb718a82af: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:29.677886 29359 log.cc:824] T 6555716b39404eb39c01555040fb4c5d P 6255d2c9742a4a71904ae5cb718a82af: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:29.678689 29358 tablet_bootstrap.cc:492] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a: No bootstrap required, opened a new log
I20260808 13:08:29.679008 29358 ts_tablet_manager.cc:1397] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.001s
I20260808 13:08:29.680262 29367 tablet_bootstrap.cc:492] T 6555716b39404eb39c01555040fb4c5d P f2d4d9a2228f405d9df8aeef8cf6d17e: Bootstrap starting.
I20260808 13:08:29.681210 29359 tablet_bootstrap.cc:492] T 6555716b39404eb39c01555040fb4c5d P 6255d2c9742a4a71904ae5cb718a82af: No bootstrap required, opened a new log
I20260808 13:08:29.681504 29359 ts_tablet_manager.cc:1397] T 6555716b39404eb39c01555040fb4c5d P 6255d2c9742a4a71904ae5cb718a82af: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 13:08:29.684101 29367 tablet_bootstrap.cc:654] T 6555716b39404eb39c01555040fb4c5d P f2d4d9a2228f405d9df8aeef8cf6d17e: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:29.685508 29367 log.cc:824] T 6555716b39404eb39c01555040fb4c5d P f2d4d9a2228f405d9df8aeef8cf6d17e: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:29.689292 29367 tablet_bootstrap.cc:492] T 6555716b39404eb39c01555040fb4c5d P f2d4d9a2228f405d9df8aeef8cf6d17e: No bootstrap required, opened a new log
I20260808 13:08:29.689586 29367 ts_tablet_manager.cc:1397] T 6555716b39404eb39c01555040fb4c5d P f2d4d9a2228f405d9df8aeef8cf6d17e: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260808 13:08:29.689949 29359 raft_consensus.cc:348] T 6555716b39404eb39c01555040fb4c5d P 6255d2c9742a4a71904ae5cb718a82af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f8c5c6225b4df1a88e9763b330fa9a" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 36015 } } peers { permanent_uuid: "6255d2c9742a4a71904ae5cb718a82af" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34283 } } peers { permanent_uuid: "f2d4d9a2228f405d9df8aeef8cf6d17e" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 42511 } }
I20260808 13:08:29.690331 29359 raft_consensus.cc:374] T 6555716b39404eb39c01555040fb4c5d P 6255d2c9742a4a71904ae5cb718a82af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:29.690446 29359 raft_consensus.cc:729] T 6555716b39404eb39c01555040fb4c5d P 6255d2c9742a4a71904ae5cb718a82af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6255d2c9742a4a71904ae5cb718a82af, State: Initialized, Role: FOLLOWER
I20260808 13:08:29.690912 29358 raft_consensus.cc:348] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f8c5c6225b4df1a88e9763b330fa9a" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 36015 } } peers { permanent_uuid: "6255d2c9742a4a71904ae5cb718a82af" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34283 } } peers { permanent_uuid: "f2d4d9a2228f405d9df8aeef8cf6d17e" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 42511 } }
I20260808 13:08:29.691130 29359 consensus_queue.cc:260] T 6555716b39404eb39c01555040fb4c5d P 6255d2c9742a4a71904ae5cb718a82af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f8c5c6225b4df1a88e9763b330fa9a" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 36015 } } peers { permanent_uuid: "6255d2c9742a4a71904ae5cb718a82af" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34283 } } peers { permanent_uuid: "f2d4d9a2228f405d9df8aeef8cf6d17e" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 42511 } }
I20260808 13:08:29.691382 29358 raft_consensus.cc:374] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:29.691529 29358 raft_consensus.cc:729] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04f8c5c6225b4df1a88e9763b330fa9a, State: Initialized, Role: FOLLOWER
I20260808 13:08:29.692868 29359 ts_tablet_manager.cc:1428] T 6555716b39404eb39c01555040fb4c5d P 6255d2c9742a4a71904ae5cb718a82af: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260808 13:08:29.692229 29358 consensus_queue.cc:260] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f8c5c6225b4df1a88e9763b330fa9a" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 36015 } } peers { permanent_uuid: "6255d2c9742a4a71904ae5cb718a82af" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34283 } } peers { permanent_uuid: "f2d4d9a2228f405d9df8aeef8cf6d17e" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 42511 } }
I20260808 13:08:29.694940 29358 ts_tablet_manager.cc:1428] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a: Time spent starting tablet: real 0.016s user 0.012s sys 0.002s
I20260808 13:08:29.695705 29323 heartbeater.cc:503] Master 127.1.124.190:37287 was elected leader, sending a full tablet report...
I20260808 13:08:29.700549 29367 raft_consensus.cc:348] T 6555716b39404eb39c01555040fb4c5d P f2d4d9a2228f405d9df8aeef8cf6d17e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f8c5c6225b4df1a88e9763b330fa9a" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 36015 } } peers { permanent_uuid: "6255d2c9742a4a71904ae5cb718a82af" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34283 } } peers { permanent_uuid: "f2d4d9a2228f405d9df8aeef8cf6d17e" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 42511 } }
I20260808 13:08:29.700959 29367 raft_consensus.cc:374] T 6555716b39404eb39c01555040fb4c5d P f2d4d9a2228f405d9df8aeef8cf6d17e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:29.701097 29367 raft_consensus.cc:729] T 6555716b39404eb39c01555040fb4c5d P f2d4d9a2228f405d9df8aeef8cf6d17e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2d4d9a2228f405d9df8aeef8cf6d17e, State: Initialized, Role: FOLLOWER
I20260808 13:08:29.701714 29367 consensus_queue.cc:260] T 6555716b39404eb39c01555040fb4c5d P f2d4d9a2228f405d9df8aeef8cf6d17e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f8c5c6225b4df1a88e9763b330fa9a" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 36015 } } peers { permanent_uuid: "6255d2c9742a4a71904ae5cb718a82af" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34283 } } peers { permanent_uuid: "f2d4d9a2228f405d9df8aeef8cf6d17e" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 42511 } }
I20260808 13:08:29.702744 29116 heartbeater.cc:503] Master 127.1.124.190:37287 was elected leader, sending a full tablet report...
I20260808 13:08:29.703480 29367 ts_tablet_manager.cc:1428] T 6555716b39404eb39c01555040fb4c5d P f2d4d9a2228f405d9df8aeef8cf6d17e: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
W20260808 13:08:29.744195 29117 tablet.cc:2369] T 6555716b39404eb39c01555040fb4c5d P f2d4d9a2228f405d9df8aeef8cf6d17e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:08:29.809891 29326 tablet.cc:2369] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:08:29.920691 29370 raft_consensus.cc:482] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:08:29.921008 29370 raft_consensus.cc:504] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f8c5c6225b4df1a88e9763b330fa9a" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 36015 } } peers { permanent_uuid: "6255d2c9742a4a71904ae5cb718a82af" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34283 } } peers { permanent_uuid: "f2d4d9a2228f405d9df8aeef8cf6d17e" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 42511 } }
I20260808 13:08:29.922797 29370 leader_election.cc:288] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6255d2c9742a4a71904ae5cb718a82af (127.1.124.129:34283), f2d4d9a2228f405d9df8aeef8cf6d17e (127.1.124.130:42511)
I20260808 13:08:29.929515 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:29.923204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:55945 (local address 127.1.124.129:34283)
0808 13:08:29.923452 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:29.923459 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:29.923518 (+ 59us) server_negotiation.cc:407] Connection header received
0808 13:08:29.923609 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:29.923614 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:29.923696 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:29.923801 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:29.924509 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.925579 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.927515 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.928570 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.928705 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:29.928757 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:29.929165 (+ 408us) server_negotiation.cc:299] Negotiation successful
0808 13:08:29.929258 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 13:08:29.929613 29328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:29.923033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:34283 (local address 127.1.124.131:55945)
0808 13:08:29.923241 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:29.923263 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:29.923407 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:29.924015 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:29.924020 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:29.924043 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:29.924326 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:29.924342 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.925727 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.925733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:29.927339 (+ 1606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:29.927352 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.928741 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.928748 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:29.928979 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:29.929022 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 13:08:29.929136 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 13:08:29.929280 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":36}
I20260808 13:08:29.930253 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:29.923715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:33265 (local address 127.1.124.130:42511)
0808 13:08:29.923917 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:29.923925 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:29.924314 (+ 389us) server_negotiation.cc:407] Connection header received
0808 13:08:29.924421 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:29.924429 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:29.924527 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:29.924599 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:29.925241 (+ 642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.926117 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.928191 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.929274 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.929422 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:29.929518 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:29.929884 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 13:08:29.929993 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 13:08:29.930939 29372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:29.923448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:42511 (local address 127.1.124.131:33265)
0808 13:08:29.924205 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:29.924219 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:29.924323 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:29.924647 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:29.924655 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:29.924680 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:29.925079 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:29.925091 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.926276 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.926280 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:29.927969 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:29.927983 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:29.929483 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:29.929489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:29.929677 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:29.929712 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 13:08:29.930429 (+ 717us) client_negotiation.cc:240] Negotiation successful
0808 13:08:29.930563 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":24,"thread_start_us":380,"threads_started":1}
I20260808 13:08:29.930773 28845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6555716b39404eb39c01555040fb4c5d" candidate_uuid: "04f8c5c6225b4df1a88e9763b330fa9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6255d2c9742a4a71904ae5cb718a82af" is_pre_election: true
I20260808 13:08:29.931418 28845 raft_consensus.cc:2454] T 6555716b39404eb39c01555040fb4c5d P 6255d2c9742a4a71904ae5cb718a82af [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04f8c5c6225b4df1a88e9763b330fa9a in term 0.
I20260808 13:08:29.932049 29047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6555716b39404eb39c01555040fb4c5d" candidate_uuid: "04f8c5c6225b4df1a88e9763b330fa9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2d4d9a2228f405d9df8aeef8cf6d17e" is_pre_election: true
I20260808 13:08:29.932262 29158 leader_election.cc:302] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04f8c5c6225b4df1a88e9763b330fa9a, 6255d2c9742a4a71904ae5cb718a82af; no voters:
W20260808 13:08:29.932667 28913 tablet.cc:2369] T 6555716b39404eb39c01555040fb4c5d P 6255d2c9742a4a71904ae5cb718a82af: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:08:29.932693 29370 raft_consensus.cc:2781] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:08:29.932667 29047 raft_consensus.cc:2454] T 6555716b39404eb39c01555040fb4c5d P f2d4d9a2228f405d9df8aeef8cf6d17e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 04f8c5c6225b4df1a88e9763b330fa9a in term 0.
I20260808 13:08:29.932843 29370 raft_consensus.cc:482] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:08:29.932956 29370 raft_consensus.cc:3037] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:29.935397 29370 raft_consensus.cc:504] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f8c5c6225b4df1a88e9763b330fa9a" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 36015 } } peers { permanent_uuid: "6255d2c9742a4a71904ae5cb718a82af" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34283 } } peers { permanent_uuid: "f2d4d9a2228f405d9df8aeef8cf6d17e" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 42511 } }
I20260808 13:08:29.936131 29370 leader_election.cc:288] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a [CANDIDATE]: Term 1 election: Requested vote from peers 6255d2c9742a4a71904ae5cb718a82af (127.1.124.129:34283), f2d4d9a2228f405d9df8aeef8cf6d17e (127.1.124.130:42511)
I20260808 13:08:29.936570 28845 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6555716b39404eb39c01555040fb4c5d" candidate_uuid: "04f8c5c6225b4df1a88e9763b330fa9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6255d2c9742a4a71904ae5cb718a82af"
I20260808 13:08:29.936659 29047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6555716b39404eb39c01555040fb4c5d" candidate_uuid: "04f8c5c6225b4df1a88e9763b330fa9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2d4d9a2228f405d9df8aeef8cf6d17e"
I20260808 13:08:29.936769 28845 raft_consensus.cc:3037] T 6555716b39404eb39c01555040fb4c5d P 6255d2c9742a4a71904ae5cb718a82af [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:29.936905 29047 raft_consensus.cc:3037] T 6555716b39404eb39c01555040fb4c5d P f2d4d9a2228f405d9df8aeef8cf6d17e [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:29.939028 28845 raft_consensus.cc:2454] T 6555716b39404eb39c01555040fb4c5d P 6255d2c9742a4a71904ae5cb718a82af [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04f8c5c6225b4df1a88e9763b330fa9a in term 1.
I20260808 13:08:29.939088 29047 raft_consensus.cc:2454] T 6555716b39404eb39c01555040fb4c5d P f2d4d9a2228f405d9df8aeef8cf6d17e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 04f8c5c6225b4df1a88e9763b330fa9a in term 1.
I20260808 13:08:29.939464 29158 leader_election.cc:302] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 04f8c5c6225b4df1a88e9763b330fa9a, 6255d2c9742a4a71904ae5cb718a82af; no voters:
I20260808 13:08:29.939770 29370 raft_consensus.cc:2781] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:08:29.940276 29370 raft_consensus.cc:686] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a [term 1 LEADER]: Becoming Leader. State: Replica: 04f8c5c6225b4df1a88e9763b330fa9a, State: Running, Role: LEADER
I20260808 13:08:29.940757 29370 consensus_queue.cc:237] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f8c5c6225b4df1a88e9763b330fa9a" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 36015 } } peers { permanent_uuid: "6255d2c9742a4a71904ae5cb718a82af" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34283 } } peers { permanent_uuid: "f2d4d9a2228f405d9df8aeef8cf6d17e" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 42511 } }
I20260808 13:08:29.945295 28626 catalog_manager.cc:5495] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a reported cstate change: term changed from 0 to 1, leader changed from <none> to 04f8c5c6225b4df1a88e9763b330fa9a (127.1.124.131). New cstate: current_term: 1 leader_uuid: "04f8c5c6225b4df1a88e9763b330fa9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "04f8c5c6225b4df1a88e9763b330fa9a" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 36015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6255d2c9742a4a71904ae5cb718a82af" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34283 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2d4d9a2228f405d9df8aeef8cf6d17e" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 42511 } health_report { overall_health: UNKNOWN } } }
I20260808 13:08:30.015880 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:30.010063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56014 (local address 127.1.124.131:36015)
0808 13:08:30.010297 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:30.010302 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:30.010323 (+ 21us) server_negotiation.cc:407] Connection header received
0808 13:08:30.010550 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:30.010572 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:30.010730 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:30.010901 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:08:30.011251 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:30.012053 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:30.013517 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:30.014075 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:30.014220 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:30.014697 (+ 477us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:08:30.015215 (+ 518us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:08:30.015338 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:30.015424 (+ 86us) server_negotiation.cc:299] Negotiation successful
0808 13:08:30.015504 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260808 13:08:30.024525 28845 raft_consensus.cc:1261] T 6555716b39404eb39c01555040fb4c5d P 6255d2c9742a4a71904ae5cb718a82af [term 1 FOLLOWER]: Refusing update from remote peer 04f8c5c6225b4df1a88e9763b330fa9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:08:30.024525 29047 raft_consensus.cc:1261] T 6555716b39404eb39c01555040fb4c5d P f2d4d9a2228f405d9df8aeef8cf6d17e [term 1 FOLLOWER]: Refusing update from remote peer 04f8c5c6225b4df1a88e9763b330fa9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:08:30.025513 29374 consensus_queue.cc:1035] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "6255d2c9742a4a71904ae5cb718a82af" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 34283 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:08:30.025897 29370 consensus_queue.cc:1035] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2d4d9a2228f405d9df8aeef8cf6d17e" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 42511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:08:30.037438 29377 mvcc.cc:205] Tried to move back new op lower bound from 7316252713050435584 to 7316252712721371136. Current Snapshot: MvccSnapshot[applied={T|T < 7316252713050435584}]
I20260808 13:08:30.053589 28627 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:08:30.053957 28627 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44062:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:08:30.058099 28627 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7559d68a45394071995a4f780503bc9f: Sending DeleteTablet for 3 replicas of tablet 6555716b39404eb39c01555040fb4c5d
I20260808 13:08:30.059337 29222 tablet_service.cc:1514] Processing DeleteTablet for tablet 6555716b39404eb39c01555040fb4c5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:08:30 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:56002
I20260808 13:08:30.059590 28815 tablet_service.cc:1514] Processing DeleteTablet for tablet 6555716b39404eb39c01555040fb4c5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:08:30 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:39316
I20260808 13:08:30.060391 29015 tablet_service.cc:1514] Processing DeleteTablet for tablet 6555716b39404eb39c01555040fb4c5d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:08:30 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:50098
I20260808 13:08:30.063673 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 28723
I20260808 13:08:30.072907 29393 tablet_replica.cc:330] T 6555716b39404eb39c01555040fb4c5d P f2d4d9a2228f405d9df8aeef8cf6d17e: stopping tablet replica
I20260808 13:08:30.073633 29393 raft_consensus.cc:2229] T 6555716b39404eb39c01555040fb4c5d P f2d4d9a2228f405d9df8aeef8cf6d17e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:08:30.074350 29393 raft_consensus.cc:2258] T 6555716b39404eb39c01555040fb4c5d P f2d4d9a2228f405d9df8aeef8cf6d17e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:08:30.077082 29393 ts_tablet_manager.cc:1905] T 6555716b39404eb39c01555040fb4c5d P f2d4d9a2228f405d9df8aeef8cf6d17e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:08:30.085927 29393 ts_tablet_manager.cc:1918] T 6555716b39404eb39c01555040fb4c5d P f2d4d9a2228f405d9df8aeef8cf6d17e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:08:30.086210 29393 log.cc:1196] T 6555716b39404eb39c01555040fb4c5d P f2d4d9a2228f405d9df8aeef8cf6d17e: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/6555716b39404eb39c01555040fb4c5d
I20260808 13:08:30.086959 29393 ts_tablet_manager.cc:1939] T 6555716b39404eb39c01555040fb4c5d P f2d4d9a2228f405d9df8aeef8cf6d17e: Deleting consensus metadata
I20260808 13:08:30.089305 28608 catalog_manager.cc:4841] TS f2d4d9a2228f405d9df8aeef8cf6d17e (127.1.124.130:42511): tablet 6555716b39404eb39c01555040fb4c5d (table test-table [id=2aa25583f347446dbb108bf5858fa86d]) successfully deleted
I20260808 13:08:30.090782 29387 tablet_replica.cc:330] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a: stopping tablet replica
I20260808 13:08:30.091475 29387 raft_consensus.cc:2229] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:08:30.092414 29387 raft_consensus.cc:2258] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:08:30.093432 29391 tablet_replica.cc:330] T 6555716b39404eb39c01555040fb4c5d P 6255d2c9742a4a71904ae5cb718a82af: stopping tablet replica
I20260808 13:08:30.093979 29391 raft_consensus.cc:2229] T 6555716b39404eb39c01555040fb4c5d P 6255d2c9742a4a71904ae5cb718a82af [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:08:30.094664 29391 raft_consensus.cc:2258] T 6555716b39404eb39c01555040fb4c5d P 6255d2c9742a4a71904ae5cb718a82af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:08:30.095503 29387 ts_tablet_manager.cc:1905] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:08:30.096942 29391 ts_tablet_manager.cc:1905] T 6555716b39404eb39c01555040fb4c5d P 6255d2c9742a4a71904ae5cb718a82af: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 13:08:30 dist-test-slave-t3q3 krb5kdc[28562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.129@KRBTEST.COM
I20260808 13:08:30.103761 29387 ts_tablet_manager.cc:1918] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:08:30.104122 29387 log.cc:1196] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/6555716b39404eb39c01555040fb4c5d
I20260808 13:08:30.104872 29387 ts_tablet_manager.cc:1939] T 6555716b39404eb39c01555040fb4c5d P 04f8c5c6225b4df1a88e9763b330fa9a: Deleting consensus metadata
I20260808 13:08:30.106057 29391 ts_tablet_manager.cc:1918] T 6555716b39404eb39c01555040fb4c5d P 6255d2c9742a4a71904ae5cb718a82af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:08:30.106354 29391 log.cc:1196] T 6555716b39404eb39c01555040fb4c5d P 6255d2c9742a4a71904ae5cb718a82af: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/6555716b39404eb39c01555040fb4c5d
I20260808 13:08:30.107223 28607 catalog_manager.cc:4841] TS 04f8c5c6225b4df1a88e9763b330fa9a (127.1.124.131:36015): tablet 6555716b39404eb39c01555040fb4c5d (table test-table [id=2aa25583f347446dbb108bf5858fa86d]) successfully deleted
I20260808 13:08:30.107388 29391 ts_tablet_manager.cc:1939] T 6555716b39404eb39c01555040fb4c5d P 6255d2c9742a4a71904ae5cb718a82af: Deleting consensus metadata
I20260808 13:08:30.109917 28608 catalog_manager.cc:4841] TS 6255d2c9742a4a71904ae5cb718a82af (127.1.124.129:34283): tablet 6555716b39404eb39c01555040fb4c5d (table test-table [id=2aa25583f347446dbb108bf5858fa86d]) successfully deleted
I20260808 13:08:30.112247 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:30.067228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39322 (local address 127.1.124.129:34283)
0808 13:08:30.067457 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:30.067466 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:30.084965 (+ 17499us) server_negotiation.cc:407] Connection header received
0808 13:08:30.085598 (+ 633us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:30.085608 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:30.085737 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:30.085830 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:30.088584 (+ 2754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:30.090264 (+ 1680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:30.093094 (+ 2830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:30.093792 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:30.093924 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:30.104330 (+ 10406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:30.104353 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:30.104374 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:30.104440 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:30.108550 (+ 4110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:30.109391 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:30.109399 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:30.109401 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:30.109559 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:30.110220 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:30.110228 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:30.110233 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:30.110774 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:30.111214 (+ 440us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:30.111499 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 13:08:30.111770 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 13:08:30.113093 28907 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:30.204495 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 28723
I20260808 13:08:30.233664 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 28923
Aug 08 13:08:30 dist-test-slave-t3q3 krb5kdc[28562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.130@KRBTEST.COM
I20260808 13:08:30.249703 29355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:30.234559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50106 (local address 127.1.124.130:42511)
0808 13:08:30.234789 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:30.234794 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:30.234814 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:08:30.234975 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:30.234980 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:30.235055 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:30.235187 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:30.236537 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:30.237292 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:30.238622 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:30.239028 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:30.239169 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:30.243639 (+ 4470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:30.243656 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:30.243672 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:30.243709 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:30.246597 (+ 2888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:30.247502 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:30.247510 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:30.247512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:30.247643 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:30.248044 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:30.248049 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:30.248052 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:30.248475 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:30.248805 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:30.249116 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 13:08:30.249349 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 13:08:30.250325 29109 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:30.343284 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 28923
I20260808 13:08:30.371358 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 29125
Aug 08 13:08:30 dist-test-slave-t3q3 krb5kdc[28562](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194507, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.131@KRBTEST.COM
I20260808 13:08:30.389904 29350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:30.372373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56020 (local address 127.1.124.131:36015)
0808 13:08:30.372615 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:30.372623 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:30.372647 (+ 24us) server_negotiation.cc:407] Connection header received
0808 13:08:30.372766 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:30.372771 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:30.372837 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:30.372975 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:30.374309 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:30.375294 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:30.377077 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:30.377579 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:30.377732 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:30.382780 (+ 5048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:30.382793 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:30.382813 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:30.382860 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:30.386378 (+ 3518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:30.387368 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:30.387376 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:30.387378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:30.387574 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:30.388188 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:30.388192 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:30.388194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:30.388626 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:30.388928 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:30.389287 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 13:08:30.389489 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 13:08:30.390801 29316 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:30.515870 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 29125
I20260808 13:08:30.540467 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 28581
I20260808 13:08:30.541611 28677 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:30.653777 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 28581
2026-08-08T13:08:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5078 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T13:08:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:08:30Z Disabled control of system clock
I20260808 13:08:30.759884 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:42495
--webserver_interface=127.1.124.190
--webserver_port=0
--builtin_ntp_servers=127.1.124.148:44647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:42495
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 13:08:31.108520 29405 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:31.108819 29405 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:31.108903 29405 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:31.116194 29405 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:08:31.116357 29405 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:31.116429 29405 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:31.116544 29405 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:08:31.116631 29405 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:08:31.126075 29405 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:44647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:42495
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:42495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:31.127470 29405 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:31.129388 29405 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:31.141800 29417 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:31.141852 29415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:31.141819 29423 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:31.142618 29405 server_base.cc:1034] running on GCE node
I20260808 13:08:31.143546 29405 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:31.145112 29405 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:31.146361 29405 hybrid_clock.cc:648] HybridClock initialized: now 1786194511146310 us; error 44 us; skew 500 ppm
I20260808 13:08:31.146776 29405 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:08:31.149266 29405 webserver.cc:460] Webserver started at http://127.1.124.190:42489/ using document root <none> and password file <none>
I20260808 13:08:31.150019 29405 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:31.150161 29405 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:31.150501 29405 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:31.153600 29405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "f2f3d726e0344523803d3be121ec5b3f"
format_stamp: "Formatted at 2026-08-08 13:08:31 on dist-test-slave-t3q3"
server_key: "88f73a303a56d81f0b5ec6aae5fa3bfb"
server_key_iv: "3a1619b11e5867694c31607c6b6cf711"
server_key_version: "encryptionkey@0"
I20260808 13:08:31.154449 29405 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "f2f3d726e0344523803d3be121ec5b3f"
format_stamp: "Formatted at 2026-08-08 13:08:31 on dist-test-slave-t3q3"
server_key: "88f73a303a56d81f0b5ec6aae5fa3bfb"
server_key_iv: "3a1619b11e5867694c31607c6b6cf711"
server_key_version: "encryptionkey@0"
I20260808 13:08:31.159894 29405 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 13:08:31.163475 29433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:31.164862 29405 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:08:31.165072 29405 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "f2f3d726e0344523803d3be121ec5b3f"
format_stamp: "Formatted at 2026-08-08 13:08:31 on dist-test-slave-t3q3"
server_key: "88f73a303a56d81f0b5ec6aae5fa3bfb"
server_key_iv: "3a1619b11e5867694c31607c6b6cf711"
server_key_version: "encryptionkey@0"
I20260808 13:08:31.165330 29405 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:31.177835 29405 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:31.178711 29405 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:31.179018 29405 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:31.198678 29405 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:42495
I20260808 13:08:31.198717 29504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:42495 every 8 connection(s)
I20260808 13:08:31.200207 29405 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:08:31.204207 29505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:31.210408 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 29405
I20260808 13:08:31.210641 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:08:31.211258 1522 external_mini_cluster.cc:1428] Setting key a2dd101a107cf2352174ec80cfd011d1
I20260808 13:08:31.215791 29505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f: Bootstrap starting.
I20260808 13:08:31.220125 29505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:31.221468 29505 log.cc:824] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:31.225054 29505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f: No bootstrap required, opened a new log
I20260808 13:08:31.229410 29511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:31.213808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41042 (local address 127.1.124.190:42495)
0808 13:08:31.215084 (+ 1276us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:31.215099 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:31.215251 (+ 152us) server_negotiation.cc:407] Connection header received
0808 13:08:31.217857 (+ 2606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:31.217890 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:31.218507 (+ 617us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:31.218974 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:31.219533 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:31.221700 (+ 2167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:31.223684 (+ 1984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:31.224822 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:31.225020 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:31.226339 (+ 1319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:31.226355 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:31.226379 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:31.227132 (+ 753us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:31.227247 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:31.227280 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:31.227396 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:31.227644 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 13:08:31.228010 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":717,"thread_start_us":445,"threads_started":1}
I20260808 13:08:31.234200 29505 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2f3d726e0344523803d3be121ec5b3f" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42495 } }
I20260808 13:08:31.234535 29505 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:31.234611 29505 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2f3d726e0344523803d3be121ec5b3f, State: Initialized, Role: FOLLOWER
I20260808 13:08:31.235188 29505 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2f3d726e0344523803d3be121ec5b3f" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42495 } }
I20260808 13:08:31.235476 29505 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:08:31.235630 29505 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:08:31.235824 29505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:31.238125 29505 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2f3d726e0344523803d3be121ec5b3f" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42495 } }
I20260808 13:08:31.238732 29505 leader_election.cc:302] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2f3d726e0344523803d3be121ec5b3f; no voters:
I20260808 13:08:31.239418 29505 leader_election.cc:288] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:08:31.239537 29513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:08:31.240295 29513 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f [term 1 LEADER]: Becoming Leader. State: Replica: f2f3d726e0344523803d3be121ec5b3f, State: Running, Role: LEADER
I20260808 13:08:31.240927 29513 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2f3d726e0344523803d3be121ec5b3f" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42495 } }
I20260808 13:08:31.241775 29505 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:08:31.243981 29516 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2f3d726e0344523803d3be121ec5b3f. Latest consensus state: current_term: 1 leader_uuid: "f2f3d726e0344523803d3be121ec5b3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2f3d726e0344523803d3be121ec5b3f" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42495 } } }
I20260808 13:08:31.243894 29514 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2f3d726e0344523803d3be121ec5b3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2f3d726e0344523803d3be121ec5b3f" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42495 } } }
I20260808 13:08:31.244447 29516 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f [sys.catalog]: This master's current role is: LEADER
I20260808 13:08:31.244529 29514 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f [sys.catalog]: This master's current role is: LEADER
I20260808 13:08:31.246235 29519 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:08:31.251835 29519 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:08:31.262053 29519 catalog_manager.cc:1347] Generated new cluster ID: 59af57786c184391b16351122628a4b0
I20260808 13:08:31.262209 29519 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:08:31.284430 29519 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:08:31.285789 29519 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:08:31.301335 29519 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f: Generated new TSK 0
I20260808 13:08:31.302492 29519 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:08:31.307832 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42495
--builtin_ntp_servers=127.1.124.148:44647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 13:08:31.656535 29540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:31.656905 29540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:31.657073 29540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:31.664727 29540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:31.664893 29540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:31.665076 29540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:08:31.674698 29540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:44647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:31.676126 29540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:31.677948 29540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:31.691658 29552 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:31.691754 29550 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:31.692320 29549 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:31.692593 29540 server_base.cc:1034] running on GCE node
I20260808 13:08:31.693257 29540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:31.694182 29540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:31.695394 29540 hybrid_clock.cc:648] HybridClock initialized: now 1786194511695349 us; error 42 us; skew 500 ppm
I20260808 13:08:31.695734 29540 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:08:31.698189 29540 webserver.cc:460] Webserver started at http://127.1.124.129:36531/ using document root <none> and password file <none>
I20260808 13:08:31.698920 29540 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:31.699016 29540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:31.699352 29540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:31.702529 29540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "4d4dfb0356fc468cbe38e72982e72e09"
format_stamp: "Formatted at 2026-08-08 13:08:31 on dist-test-slave-t3q3"
server_key: "5b905165018792b29ba4c6e121b3ff4f"
server_key_iv: "d11891e2bbcef74e801949e7f28c08ec"
server_key_version: "encryptionkey@0"
I20260808 13:08:31.703395 29540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "4d4dfb0356fc468cbe38e72982e72e09"
format_stamp: "Formatted at 2026-08-08 13:08:31 on dist-test-slave-t3q3"
server_key: "5b905165018792b29ba4c6e121b3ff4f"
server_key_iv: "d11891e2bbcef74e801949e7f28c08ec"
server_key_version: "encryptionkey@0"
I20260808 13:08:31.708724 29540 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 13:08:31.712440 29561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:31.713893 29540 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:08:31.714102 29540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "4d4dfb0356fc468cbe38e72982e72e09"
format_stamp: "Formatted at 2026-08-08 13:08:31 on dist-test-slave-t3q3"
server_key: "5b905165018792b29ba4c6e121b3ff4f"
server_key_iv: "d11891e2bbcef74e801949e7f28c08ec"
server_key_version: "encryptionkey@0"
I20260808 13:08:31.714366 29540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:31.730933 29540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:31.732209 29540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:31.732561 29540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:31.733637 29540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:08:31.735163 29540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:08:31.735306 29540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:31.735445 29540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:08:31.735539 29540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:31.771592 29540 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:42445
I20260808 13:08:31.771687 29737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:42445 every 8 connection(s)
I20260808 13:08:31.773061 29540 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:08:31.781709 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 29540
I20260808 13:08:31.781915 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:08:31.782485 1522 external_mini_cluster.cc:1428] Setting key 71ba7b4f2badb898b18eeccb0b99d565
I20260808 13:08:31.785761 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42495
--builtin_ntp_servers=127.1.124.148:44647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 13:08:31.787317 29741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:31.775784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:39269 (local address 127.1.124.190:42495)
0808 13:08:31.776280 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:31.776287 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:31.777720 (+ 1433us) server_negotiation.cc:407] Connection header received
0808 13:08:31.779543 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:31.779548 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:31.779617 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:31.779719 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:31.781014 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:31.782084 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:31.783956 (+ 1872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:31.784632 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:31.784746 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:31.786502 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:31.786507 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:31.786511 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:31.786531 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:31.786574 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:31.786586 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:31.786669 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:31.786960 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 13:08:31.787044 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":218,"threads_started":1}
I20260808 13:08:31.788954 29742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:31.776527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:42495 (local address 127.1.124.129:39269)
0808 13:08:31.777546 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:31.777589 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:31.779281 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:31.780017 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:31.780026 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:31.780192 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:08:31.780812 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:31.780847 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:31.782290 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:31.782294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:31.783773 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:31.783788 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:31.784901 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:31.784909 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:31.785235 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:31.786081 (+ 846us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:08:31.786086 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:08:31.786182 (+ 96us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:08:31.786299 (+ 117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:08:31.786693 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:08:31.786707 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:08:31.786720 (+ 13us) client_negotiation.cc:769] Sending connection context
0808 13:08:31.786932 (+ 212us) client_negotiation.cc:240] Negotiation successful
0808 13:08:31.787425 (+ 493us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":313,"threads_started":1}
I20260808 13:08:31.790740 29739 heartbeater.cc:347] Connected to a master server at 127.1.124.190:42495
I20260808 13:08:31.791100 29739 heartbeater.cc:464] Registering TS with master...
I20260808 13:08:31.791949 29739 heartbeater.cc:511] Master 127.1.124.190:42495 requested a full tablet report, sending...
I20260808 13:08:31.794226 29458 ts_manager.cc:194] Registered new tserver with Master: 4d4dfb0356fc468cbe38e72982e72e09 (127.1.124.129:42445)
I20260808 13:08:31.795810 29458 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.124.129:39269
W20260808 13:08:32.134377 29744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:32.134752 29744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:32.134918 29744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:32.142760 29744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:32.142925 29744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:32.143160 29744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:08:32.152925 29744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:44647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:32.154387 29744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:32.156406 29744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:32.170873 29750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:32.170855 29751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:32.170971 29757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:32.172365 29744 server_base.cc:1034] running on GCE node
I20260808 13:08:32.173090 29744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:32.174151 29744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:32.175413 29744 hybrid_clock.cc:648] HybridClock initialized: now 1786194512175351 us; error 58 us; skew 500 ppm
I20260808 13:08:32.175782 29744 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:08:32.178221 29744 webserver.cc:460] Webserver started at http://127.1.124.130:40899/ using document root <none> and password file <none>
I20260808 13:08:32.178982 29744 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:32.179123 29744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:32.179458 29744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:32.182538 29744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "401171b3bbcf40028b6f6c2f24c3c7f2"
format_stamp: "Formatted at 2026-08-08 13:08:32 on dist-test-slave-t3q3"
server_key: "dc2aad671d8f46c86425b100536a5a8a"
server_key_iv: "dbde2101671a02fe7eab2fc04ec637f4"
server_key_version: "encryptionkey@0"
I20260808 13:08:32.183350 29744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "401171b3bbcf40028b6f6c2f24c3c7f2"
format_stamp: "Formatted at 2026-08-08 13:08:32 on dist-test-slave-t3q3"
server_key: "dc2aad671d8f46c86425b100536a5a8a"
server_key_iv: "dbde2101671a02fe7eab2fc04ec637f4"
server_key_version: "encryptionkey@0"
I20260808 13:08:32.188690 29744 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.007s
I20260808 13:08:32.192390 29765 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:32.193694 29744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:08:32.193876 29744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "401171b3bbcf40028b6f6c2f24c3c7f2"
format_stamp: "Formatted at 2026-08-08 13:08:32 on dist-test-slave-t3q3"
server_key: "dc2aad671d8f46c86425b100536a5a8a"
server_key_iv: "dbde2101671a02fe7eab2fc04ec637f4"
server_key_version: "encryptionkey@0"
I20260808 13:08:32.194110 29744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:32.210629 29744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:32.211448 29744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:32.211795 29744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:32.212976 29744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:08:32.214447 29744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:08:32.214587 29744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:32.214723 29744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:08:32.214812 29744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:32.249639 29744 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:41673
I20260808 13:08:32.249686 29923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:41673 every 8 connection(s)
I20260808 13:08:32.250975 29744 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:08:32.259819 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 29744
I20260808 13:08:32.260020 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:08:32.260480 1522 external_mini_cluster.cc:1428] Setting key f600874d37a56ce24e0f9b2a794070a0
I20260808 13:08:32.263334 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42495
--builtin_ntp_servers=127.1.124.148:44647
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 13:08:32.264037 29741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:32.253635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:46771 (local address 127.1.124.190:42495)
0808 13:08:32.253828 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:32.253832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:32.255134 (+ 1302us) server_negotiation.cc:407] Connection header received
0808 13:08:32.257049 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:32.257054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:32.257142 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:32.257266 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:32.258382 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:32.259104 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:32.260919 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:32.261489 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:32.261633 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:32.263087 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:32.263092 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:32.263095 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:32.263112 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:32.263142 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:32.263163 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:32.263262 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:32.263549 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 13:08:32.263655 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 13:08:32.264935 29927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:32.254083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:42495 (local address 127.1.124.130:46771)
0808 13:08:32.254966 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:32.255004 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:32.256769 (+ 1765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:32.257518 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:32.257526 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:32.257610 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:08:32.258204 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:32.258221 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:32.259273 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:32.259278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:32.260774 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:32.260788 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:32.261741 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:32.261750 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:32.261977 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:32.262776 (+ 799us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:08:32.262779 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:08:32.262817 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:08:32.262895 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:08:32.263264 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:08:32.263278 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:08:32.263288 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 13:08:32.263488 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 13:08:32.263828 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":266,"threads_started":1}
I20260808 13:08:32.266335 29925 heartbeater.cc:347] Connected to a master server at 127.1.124.190:42495
I20260808 13:08:32.266645 29925 heartbeater.cc:464] Registering TS with master...
I20260808 13:08:32.267369 29925 heartbeater.cc:511] Master 127.1.124.190:42495 requested a full tablet report, sending...
I20260808 13:08:32.268862 29458 ts_manager.cc:194] Registered new tserver with Master: 401171b3bbcf40028b6f6c2f24c3c7f2 (127.1.124.130:41673)
I20260808 13:08:32.269850 29458 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.124.130:46771
W20260808 13:08:32.613646 29929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:32.614014 29929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:32.614117 29929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:32.621363 29929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:32.621460 29929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:32.621565 29929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:08:32.630507 29929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:44647
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42495
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:32.631688 29929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:32.633423 29929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:32.647087 29937 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:32.647039 29938 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:32.647055 29940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:32.648188 29929 server_base.cc:1034] running on GCE node
I20260808 13:08:32.648897 29929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:32.649883 29929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:32.651137 29929 hybrid_clock.cc:648] HybridClock initialized: now 1786194512651077 us; error 68 us; skew 500 ppm
I20260808 13:08:32.651435 29929 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:08:32.654019 29929 webserver.cc:460] Webserver started at http://127.1.124.131:39617/ using document root <none> and password file <none>
I20260808 13:08:32.654740 29929 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:32.654832 29929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:32.655113 29929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:32.658070 29929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "1993589d7d8a4287930ff4d2bc25aab5"
format_stamp: "Formatted at 2026-08-08 13:08:32 on dist-test-slave-t3q3"
server_key: "484a72ce7c6faadcf606121b942e56ee"
server_key_iv: "653c49a17ec83f0d851478c97124ea7f"
server_key_version: "encryptionkey@0"
I20260808 13:08:32.658833 29929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "1993589d7d8a4287930ff4d2bc25aab5"
format_stamp: "Formatted at 2026-08-08 13:08:32 on dist-test-slave-t3q3"
server_key: "484a72ce7c6faadcf606121b942e56ee"
server_key_iv: "653c49a17ec83f0d851478c97124ea7f"
server_key_version: "encryptionkey@0"
I20260808 13:08:32.663954 29929 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 13:08:32.667693 29949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:32.669312 29929 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:08:32.669481 29929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "1993589d7d8a4287930ff4d2bc25aab5"
format_stamp: "Formatted at 2026-08-08 13:08:32 on dist-test-slave-t3q3"
server_key: "484a72ce7c6faadcf606121b942e56ee"
server_key_iv: "653c49a17ec83f0d851478c97124ea7f"
server_key_version: "encryptionkey@0"
I20260808 13:08:32.669766 29929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:32.697582 29929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:32.698510 29929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:32.698841 29929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:32.700035 29929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:08:32.701603 29929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:08:32.701694 29929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:32.701772 29929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:08:32.701833 29929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:32.744797 29929 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:43633
I20260808 13:08:32.744913 30109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:43633 every 8 connection(s)
I20260808 13:08:32.746205 29929 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:08:32.746598 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 29929
I20260808 13:08:32.746866 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:08:32.747447 1522 external_mini_cluster.cc:1428] Setting key 626058e4564580f6dc2c3831be047cc4
I20260808 13:08:32.759570 29741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:32.749326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:52205 (local address 127.1.124.190:42495)
0808 13:08:32.749524 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:32.749529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:32.751280 (+ 1751us) server_negotiation.cc:407] Connection header received
0808 13:08:32.753240 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:32.753244 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:32.753307 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:32.753389 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:32.754491 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:32.755244 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:32.756992 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:32.757444 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:32.757543 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:32.758885 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:32.758890 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:32.758893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:32.758911 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:32.758942 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:32.758953 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:32.759023 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:32.759256 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 13:08:32.759332 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 13:08:32.760701 30113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:32.750019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:42495 (local address 127.1.124.131:52205)
0808 13:08:32.751126 (+ 1107us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:32.751174 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:32.752979 (+ 1805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:32.753599 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:32.753607 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:32.753701 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:08:32.754324 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:32.754348 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:32.755402 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:32.755406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:32.756856 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:32.756869 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:32.757650 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:32.757655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:32.757858 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:32.758557 (+ 699us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:08:32.758560 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:08:32.758606 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:08:32.758686 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:08:32.759054 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:08:32.759063 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:08:32.759069 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 13:08:32.759224 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 13:08:32.759546 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":361,"threads_started":1}
I20260808 13:08:32.762413 30110 heartbeater.cc:347] Connected to a master server at 127.1.124.190:42495
I20260808 13:08:32.762759 30110 heartbeater.cc:464] Registering TS with master...
I20260808 13:08:32.763610 30110 heartbeater.cc:511] Master 127.1.124.190:42495 requested a full tablet report, sending...
I20260808 13:08:32.765075 29457 ts_manager.cc:194] Registered new tserver with Master: 1993589d7d8a4287930ff4d2bc25aab5 (127.1.124.131:43633)
I20260808 13:08:32.765991 29457 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.124.131:52205
I20260808 13:08:32.776335 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:08:32.787225 29741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:32.780801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41048 (local address 127.1.124.190:42495)
0808 13:08:32.781063 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:32.781073 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:32.781472 (+ 399us) server_negotiation.cc:407] Connection header received
0808 13:08:32.781579 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:32.781585 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:32.781674 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:32.781767 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:32.782281 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:32.783173 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:32.784770 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:32.785291 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:32.785399 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:32.786476 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:32.786481 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:32.786485 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:32.786504 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:32.786537 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:32.786548 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:32.786621 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:32.786759 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 13:08:32.786855 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 13:08:32.791383 29457 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:41048:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:08:32.798555 29457 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:08:32.799312 29739 heartbeater.cc:503] Master 127.1.124.190:42495 was elected leader, sending a full tablet report...
I20260808 13:08:32.833876 30131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:32.811442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57098 (local address 127.1.124.131:43633)
0808 13:08:32.826401 (+ 14959us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:32.826408 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:32.826437 (+ 29us) server_negotiation.cc:407] Connection header received
0808 13:08:32.826526 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:32.826532 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:32.826715 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:32.826896 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:32.827686 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:32.828946 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:32.831239 (+ 2293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:32.832638 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:32.832766 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:32.833066 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:32.833318 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 13:08:32.833496 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14798,"thread_start_us":14623,"threads_started":1}
I20260808 13:08:32.833896 30130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:32.811173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:43633 (local address 127.0.0.1:57098)
0808 13:08:32.812720 (+ 1547us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:32.812767 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:32.812936 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:32.826913 (+ 13977us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:32.826918 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:32.826948 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:32.827448 (+ 500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:32.827462 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:32.829105 (+ 1643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:32.829111 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:32.831046 (+ 1935us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:32.831060 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:32.832839 (+ 1779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:32.832846 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:32.833063 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:32.833127 (+ 64us) client_negotiation.cc:769] Sending connection context
0808 13:08:32.833258 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 13:08:32.833444 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1291,"mutex_wait_us":59,"thread_start_us":1157,"threads_started":1}
I20260808 13:08:32.834486 30132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:32.813302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:42445 (local address 127.0.0.1:50534)
0808 13:08:32.825959 (+ 12657us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:32.825983 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:32.826112 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:32.827098 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:32.827102 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:32.827131 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:32.827643 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:32.827656 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:32.829370 (+ 1714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:32.829375 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:32.831688 (+ 2313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:32.831705 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:32.833651 (+ 1946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:32.833657 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:32.833844 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:32.833891 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 13:08:32.833997 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 13:08:32.834131 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12525,"mutex_wait_us":79,"thread_start_us":11746,"threads_started":1}
I20260808 13:08:32.834949 30137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:32.813417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50534 (local address 127.1.124.129:42445)
0808 13:08:32.825035 (+ 11618us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:32.825041 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:32.826252 (+ 1211us) server_negotiation.cc:407] Connection header received
0808 13:08:32.826360 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:32.826370 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:32.826593 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:32.826797 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:32.827815 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:32.829202 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:32.831886 (+ 2684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:32.833137 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:32.833273 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:32.833525 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:32.834396 (+ 871us) server_negotiation.cc:299] Negotiation successful
0808 13:08:32.834597 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11441,"thread_start_us":11310,"threads_started":1}
I20260808 13:08:32.834959 30133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:32.812051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42292 (local address 127.1.124.130:41673)
0808 13:08:32.821265 (+ 9214us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:32.821288 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:32.824379 (+ 3091us) server_negotiation.cc:407] Connection header received
0808 13:08:32.824648 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:32.824661 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:32.824999 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:32.825260 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:32.826809 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:32.828534 (+ 1725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:32.832169 (+ 3635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:32.833529 (+ 1360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:32.833662 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:32.833891 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:32.834449 (+ 558us) server_negotiation.cc:299] Negotiation successful
0808 13:08:32.834625 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9017,"thread_start_us":8876,"threads_started":1}
I20260808 13:08:32.835577 30136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:32.811434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:41673 (local address 127.0.0.1:42292)
0808 13:08:32.824254 (+ 12820us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:32.824281 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:32.824450 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:32.825299 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:32.825311 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:32.825354 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:32.826605 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:32.826622 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:32.828681 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:32.828686 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:32.831943 (+ 3257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:32.831960 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:32.834042 (+ 2082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:32.834051 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:32.834231 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:32.834261 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 13:08:32.835210 (+ 949us) client_negotiation.cc:240] Negotiation successful
0808 13:08:32.835344 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12647,"mutex_wait_us":685,"thread_start_us":14262,"threads_started":1}
I20260808 13:08:32.838445 29640 tablet_service.cc:1467] Processing CreateTablet for tablet 8f416726a39948b4b634b696e4608443 (DEFAULT_TABLE table=test-table [id=1d7ea092e9054e139d4156067c0df0e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:08:32.838488 30019 tablet_service.cc:1467] Processing CreateTablet for tablet 8f416726a39948b4b634b696e4608443 (DEFAULT_TABLE table=test-table [id=1d7ea092e9054e139d4156067c0df0e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:08:32.838922 29831 tablet_service.cc:1467] Processing CreateTablet for tablet 8f416726a39948b4b634b696e4608443 (DEFAULT_TABLE table=test-table [id=1d7ea092e9054e139d4156067c0df0e6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:08:32.839802 29640 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f416726a39948b4b634b696e4608443. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:32.839941 30019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f416726a39948b4b634b696e4608443. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:32.841032 29831 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8f416726a39948b4b634b696e4608443. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:32.851373 30139 tablet_bootstrap.cc:492] T 8f416726a39948b4b634b696e4608443 P 4d4dfb0356fc468cbe38e72982e72e09: Bootstrap starting.
I20260808 13:08:32.854910 30141 tablet_bootstrap.cc:492] T 8f416726a39948b4b634b696e4608443 P 1993589d7d8a4287930ff4d2bc25aab5: Bootstrap starting.
I20260808 13:08:32.856107 30139 tablet_bootstrap.cc:654] T 8f416726a39948b4b634b696e4608443 P 4d4dfb0356fc468cbe38e72982e72e09: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:32.856621 30142 tablet_bootstrap.cc:492] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2: Bootstrap starting.
I20260808 13:08:32.858070 30139 log.cc:824] T 8f416726a39948b4b634b696e4608443 P 4d4dfb0356fc468cbe38e72982e72e09: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:32.858698 30141 tablet_bootstrap.cc:654] T 8f416726a39948b4b634b696e4608443 P 1993589d7d8a4287930ff4d2bc25aab5: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:32.859795 30142 tablet_bootstrap.cc:654] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:32.859757 30141 log.cc:824] T 8f416726a39948b4b634b696e4608443 P 1993589d7d8a4287930ff4d2bc25aab5: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:32.861088 30142 log.cc:824] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:32.861402 30139 tablet_bootstrap.cc:492] T 8f416726a39948b4b634b696e4608443 P 4d4dfb0356fc468cbe38e72982e72e09: No bootstrap required, opened a new log
I20260808 13:08:32.861690 30139 ts_tablet_manager.cc:1397] T 8f416726a39948b4b634b696e4608443 P 4d4dfb0356fc468cbe38e72982e72e09: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260808 13:08:32.862937 30141 tablet_bootstrap.cc:492] T 8f416726a39948b4b634b696e4608443 P 1993589d7d8a4287930ff4d2bc25aab5: No bootstrap required, opened a new log
I20260808 13:08:32.863238 30141 ts_tablet_manager.cc:1397] T 8f416726a39948b4b634b696e4608443 P 1993589d7d8a4287930ff4d2bc25aab5: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260808 13:08:32.863680 30142 tablet_bootstrap.cc:492] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2: No bootstrap required, opened a new log
I20260808 13:08:32.863966 30142 ts_tablet_manager.cc:1397] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.007s
I20260808 13:08:32.871410 30142 raft_consensus.cc:348] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1993589d7d8a4287930ff4d2bc25aab5" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 43633 } } peers { permanent_uuid: "401171b3bbcf40028b6f6c2f24c3c7f2" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41673 } } peers { permanent_uuid: "4d4dfb0356fc468cbe38e72982e72e09" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42445 } }
I20260808 13:08:32.871820 30142 raft_consensus.cc:374] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:32.871989 30142 raft_consensus.cc:729] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 401171b3bbcf40028b6f6c2f24c3c7f2, State: Initialized, Role: FOLLOWER
I20260808 13:08:32.872685 30142 consensus_queue.cc:260] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1993589d7d8a4287930ff4d2bc25aab5" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 43633 } } peers { permanent_uuid: "401171b3bbcf40028b6f6c2f24c3c7f2" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41673 } } peers { permanent_uuid: "4d4dfb0356fc468cbe38e72982e72e09" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42445 } }
I20260808 13:08:32.873549 30139 raft_consensus.cc:348] T 8f416726a39948b4b634b696e4608443 P 4d4dfb0356fc468cbe38e72982e72e09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1993589d7d8a4287930ff4d2bc25aab5" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 43633 } } peers { permanent_uuid: "401171b3bbcf40028b6f6c2f24c3c7f2" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41673 } } peers { permanent_uuid: "4d4dfb0356fc468cbe38e72982e72e09" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42445 } }
I20260808 13:08:32.873899 29925 heartbeater.cc:503] Master 127.1.124.190:42495 was elected leader, sending a full tablet report...
I20260808 13:08:32.874078 30139 raft_consensus.cc:374] T 8f416726a39948b4b634b696e4608443 P 4d4dfb0356fc468cbe38e72982e72e09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:32.874248 30139 raft_consensus.cc:729] T 8f416726a39948b4b634b696e4608443 P 4d4dfb0356fc468cbe38e72982e72e09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d4dfb0356fc468cbe38e72982e72e09, State: Initialized, Role: FOLLOWER
I20260808 13:08:32.874229 30141 raft_consensus.cc:348] T 8f416726a39948b4b634b696e4608443 P 1993589d7d8a4287930ff4d2bc25aab5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1993589d7d8a4287930ff4d2bc25aab5" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 43633 } } peers { permanent_uuid: "401171b3bbcf40028b6f6c2f24c3c7f2" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41673 } } peers { permanent_uuid: "4d4dfb0356fc468cbe38e72982e72e09" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42445 } }
I20260808 13:08:32.874712 30141 raft_consensus.cc:374] T 8f416726a39948b4b634b696e4608443 P 1993589d7d8a4287930ff4d2bc25aab5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:32.874836 30142 ts_tablet_manager.cc:1428] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260808 13:08:32.874846 30141 raft_consensus.cc:729] T 8f416726a39948b4b634b696e4608443 P 1993589d7d8a4287930ff4d2bc25aab5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1993589d7d8a4287930ff4d2bc25aab5, State: Initialized, Role: FOLLOWER
I20260808 13:08:32.875092 30139 consensus_queue.cc:260] T 8f416726a39948b4b634b696e4608443 P 4d4dfb0356fc468cbe38e72982e72e09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1993589d7d8a4287930ff4d2bc25aab5" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 43633 } } peers { permanent_uuid: "401171b3bbcf40028b6f6c2f24c3c7f2" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41673 } } peers { permanent_uuid: "4d4dfb0356fc468cbe38e72982e72e09" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42445 } }
I20260808 13:08:32.875411 30141 consensus_queue.cc:260] T 8f416726a39948b4b634b696e4608443 P 1993589d7d8a4287930ff4d2bc25aab5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1993589d7d8a4287930ff4d2bc25aab5" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 43633 } } peers { permanent_uuid: "401171b3bbcf40028b6f6c2f24c3c7f2" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41673 } } peers { permanent_uuid: "4d4dfb0356fc468cbe38e72982e72e09" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42445 } }
I20260808 13:08:32.877391 30141 ts_tablet_manager.cc:1428] T 8f416726a39948b4b634b696e4608443 P 1993589d7d8a4287930ff4d2bc25aab5: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260808 13:08:32.878270 30110 heartbeater.cc:503] Master 127.1.124.190:42495 was elected leader, sending a full tablet report...
I20260808 13:08:32.883208 30139 ts_tablet_manager.cc:1428] T 8f416726a39948b4b634b696e4608443 P 4d4dfb0356fc468cbe38e72982e72e09: Time spent starting tablet: real 0.021s user 0.011s sys 0.008s
W20260808 13:08:32.998311 30112 tablet.cc:2369] T 8f416726a39948b4b634b696e4608443 P 1993589d7d8a4287930ff4d2bc25aab5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:08:33.003052 29926 tablet.cc:2369] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:08:33.011569 30146 raft_consensus.cc:482] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:08:33.011890 30146 raft_consensus.cc:504] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1993589d7d8a4287930ff4d2bc25aab5" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 43633 } } peers { permanent_uuid: "401171b3bbcf40028b6f6c2f24c3c7f2" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41673 } } peers { permanent_uuid: "4d4dfb0356fc468cbe38e72982e72e09" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42445 } }
I20260808 13:08:33.013652 30146 leader_election.cc:288] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1993589d7d8a4287930ff4d2bc25aab5 (127.1.124.131:43633), 4d4dfb0356fc468cbe38e72982e72e09 (127.1.124.129:42445)
I20260808 13:08:33.020597 30131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:33.014165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:46529 (local address 127.1.124.131:43633)
0808 13:08:33.014429 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:33.014438 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:33.014827 (+ 389us) server_negotiation.cc:407] Connection header received
0808 13:08:33.015101 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:33.015106 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:33.015207 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:33.015306 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:33.015898 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:33.016694 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:33.018690 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:33.019630 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:33.019814 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:33.019887 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:33.020231 (+ 344us) server_negotiation.cc:299] Negotiation successful
0808 13:08:33.020351 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 13:08:33.020704 30137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:33.015025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:51841 (local address 127.1.124.129:42445)
0808 13:08:33.015274 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:33.015280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:33.015547 (+ 267us) server_negotiation.cc:407] Connection header received
0808 13:08:33.015763 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:33.015771 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:33.015843 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:33.015942 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:33.016494 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:33.017159 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:33.018980 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:33.019788 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:33.019960 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:33.020031 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:33.020345 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 13:08:33.020463 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260808 13:08:33.020746 30150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:33.014020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:43633 (local address 127.1.124.130:46529)
0808 13:08:33.014718 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:33.014734 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:33.014911 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:33.015359 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:33.015364 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:33.015388 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:33.015754 (+ 366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:33.015763 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:33.016830 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:33.016835 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:33.018503 (+ 1668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:33.018515 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:33.019819 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:33.019825 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:33.020024 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:33.020048 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 13:08:33.020207 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 13:08:33.020331 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":576,"mutex_wait_us":21,"thread_start_us":396,"threads_started":1}
I20260808 13:08:33.020839 30151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:33.014860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:42445 (local address 127.1.124.130:51841)
0808 13:08:33.015441 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:33.015460 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:33.015624 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:33.015958 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:33.015962 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:33.015984 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:33.016346 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:33.016355 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:33.017303 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:33.017310 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:33.018802 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:33.018813 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:33.019980 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:33.019986 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:33.020188 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:33.020210 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 13:08:33.020315 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 13:08:33.020435 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":22,"thread_start_us":300,"threads_started":1}
I20260808 13:08:33.021770 29673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f416726a39948b4b634b696e4608443" candidate_uuid: "401171b3bbcf40028b6f6c2f24c3c7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4dfb0356fc468cbe38e72982e72e09" is_pre_election: true
I20260808 13:08:33.021768 30049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f416726a39948b4b634b696e4608443" candidate_uuid: "401171b3bbcf40028b6f6c2f24c3c7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1993589d7d8a4287930ff4d2bc25aab5" is_pre_election: true
I20260808 13:08:33.022286 30049 raft_consensus.cc:2454] T 8f416726a39948b4b634b696e4608443 P 1993589d7d8a4287930ff4d2bc25aab5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 401171b3bbcf40028b6f6c2f24c3c7f2 in term 0.
I20260808 13:08:33.022286 29673 raft_consensus.cc:2454] T 8f416726a39948b4b634b696e4608443 P 4d4dfb0356fc468cbe38e72982e72e09 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 401171b3bbcf40028b6f6c2f24c3c7f2 in term 0.
I20260808 13:08:33.022902 29769 leader_election.cc:302] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1993589d7d8a4287930ff4d2bc25aab5, 401171b3bbcf40028b6f6c2f24c3c7f2; no voters:
I20260808 13:08:33.023272 30146 raft_consensus.cc:2781] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:08:33.023419 30146 raft_consensus.cc:482] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:08:33.023524 30146 raft_consensus.cc:3037] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2 [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:08:33.025508 29740 tablet.cc:2369] T 8f416726a39948b4b634b696e4608443 P 4d4dfb0356fc468cbe38e72982e72e09: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:08:33.026115 30146 raft_consensus.cc:504] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1993589d7d8a4287930ff4d2bc25aab5" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 43633 } } peers { permanent_uuid: "401171b3bbcf40028b6f6c2f24c3c7f2" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41673 } } peers { permanent_uuid: "4d4dfb0356fc468cbe38e72982e72e09" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42445 } }
I20260808 13:08:33.026686 30146 leader_election.cc:288] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2 [CANDIDATE]: Term 1 election: Requested vote from peers 1993589d7d8a4287930ff4d2bc25aab5 (127.1.124.131:43633), 4d4dfb0356fc468cbe38e72982e72e09 (127.1.124.129:42445)
I20260808 13:08:33.027177 30049 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f416726a39948b4b634b696e4608443" candidate_uuid: "401171b3bbcf40028b6f6c2f24c3c7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1993589d7d8a4287930ff4d2bc25aab5"
I20260808 13:08:33.027240 29673 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8f416726a39948b4b634b696e4608443" candidate_uuid: "401171b3bbcf40028b6f6c2f24c3c7f2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4d4dfb0356fc468cbe38e72982e72e09"
I20260808 13:08:33.027374 30049 raft_consensus.cc:3037] T 8f416726a39948b4b634b696e4608443 P 1993589d7d8a4287930ff4d2bc25aab5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:33.027442 29673 raft_consensus.cc:3037] T 8f416726a39948b4b634b696e4608443 P 4d4dfb0356fc468cbe38e72982e72e09 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:33.029620 30049 raft_consensus.cc:2454] T 8f416726a39948b4b634b696e4608443 P 1993589d7d8a4287930ff4d2bc25aab5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 401171b3bbcf40028b6f6c2f24c3c7f2 in term 1.
I20260808 13:08:33.029680 29673 raft_consensus.cc:2454] T 8f416726a39948b4b634b696e4608443 P 4d4dfb0356fc468cbe38e72982e72e09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 401171b3bbcf40028b6f6c2f24c3c7f2 in term 1.
I20260808 13:08:33.030042 29769 leader_election.cc:302] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1993589d7d8a4287930ff4d2bc25aab5, 401171b3bbcf40028b6f6c2f24c3c7f2; no voters:
I20260808 13:08:33.030357 30146 raft_consensus.cc:2781] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:08:33.030807 30146 raft_consensus.cc:686] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2 [term 1 LEADER]: Becoming Leader. State: Replica: 401171b3bbcf40028b6f6c2f24c3c7f2, State: Running, Role: LEADER
I20260808 13:08:33.031271 30146 consensus_queue.cc:237] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1993589d7d8a4287930ff4d2bc25aab5" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 43633 } } peers { permanent_uuid: "401171b3bbcf40028b6f6c2f24c3c7f2" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41673 } } peers { permanent_uuid: "4d4dfb0356fc468cbe38e72982e72e09" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42445 } }
I20260808 13:08:33.036242 29457 catalog_manager.cc:5495] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 401171b3bbcf40028b6f6c2f24c3c7f2 (127.1.124.130). New cstate: current_term: 1 leader_uuid: "401171b3bbcf40028b6f6c2f24c3c7f2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1993589d7d8a4287930ff4d2bc25aab5" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 43633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "401171b3bbcf40028b6f6c2f24c3c7f2" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 41673 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4d4dfb0356fc468cbe38e72982e72e09" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42445 } health_report { overall_health: UNKNOWN } } }
I20260808 13:08:33.062506 30133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:33.057436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42298 (local address 127.1.124.130:41673)
0808 13:08:33.057625 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:33.057629 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:33.057646 (+ 17us) server_negotiation.cc:407] Connection header received
0808 13:08:33.057838 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:33.057842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:33.057938 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:33.058065 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:08:33.058390 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:33.059100 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:33.060565 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:33.060983 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:33.061076 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:33.061540 (+ 464us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:08:33.062052 (+ 512us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:08:33.062169 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:33.062231 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 13:08:33.062292 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 13:08:33.070840 30049 raft_consensus.cc:1261] T 8f416726a39948b4b634b696e4608443 P 1993589d7d8a4287930ff4d2bc25aab5 [term 1 FOLLOWER]: Refusing update from remote peer 401171b3bbcf40028b6f6c2f24c3c7f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:08:33.070834 29673 raft_consensus.cc:1261] T 8f416726a39948b4b634b696e4608443 P 4d4dfb0356fc468cbe38e72982e72e09 [term 1 FOLLOWER]: Refusing update from remote peer 401171b3bbcf40028b6f6c2f24c3c7f2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:08:33.071869 30152 consensus_queue.cc:1035] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1993589d7d8a4287930ff4d2bc25aab5" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 43633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:08:33.072326 30146 consensus_queue.cc:1035] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4d4dfb0356fc468cbe38e72982e72e09" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 42445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:08:33.086480 30156 mvcc.cc:205] Tried to move back new op lower bound from 7316252725527379968 to 7316252725380165632. Current Snapshot: MvccSnapshot[applied={T|T < 7316252725527379968}]
I20260808 13:08:33.098899 29457 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:41048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:08:33.099217 29457 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:41048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:08:33.103497 29457 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f2f3d726e0344523803d3be121ec5b3f: Sending DeleteTablet for 3 replicas of tablet 8f416726a39948b4b634b696e4608443
I20260808 13:08:33.104790 29831 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f416726a39948b4b634b696e4608443 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:08:33 UTC) from {username='slave'} at 127.0.0.1:42292
I20260808 13:08:33.104880 29640 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f416726a39948b4b634b696e4608443 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:08:33 UTC) from {username='slave'} at 127.0.0.1:50534
I20260808 13:08:33.105406 30019 tablet_service.cc:1514] Processing DeleteTablet for tablet 8f416726a39948b4b634b696e4608443 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:08:33 UTC) from {username='slave'} at 127.0.0.1:57098
I20260808 13:08:33.107641 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 29540
I20260808 13:08:33.120772 30171 tablet_replica.cc:330] T 8f416726a39948b4b634b696e4608443 P 1993589d7d8a4287930ff4d2bc25aab5: stopping tablet replica
I20260808 13:08:33.120812 30169 tablet_replica.cc:330] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2: stopping tablet replica
I20260808 13:08:33.121328 30169 raft_consensus.cc:2229] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:08:33.121331 30171 raft_consensus.cc:2229] T 8f416726a39948b4b634b696e4608443 P 1993589d7d8a4287930ff4d2bc25aab5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:08:33.121917 30171 raft_consensus.cc:2258] T 8f416726a39948b4b634b696e4608443 P 1993589d7d8a4287930ff4d2bc25aab5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:08:33.122270 30169 raft_consensus.cc:2258] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:08:33.125375 30169 ts_tablet_manager.cc:1905] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:08:33.126852 30171 ts_tablet_manager.cc:1905] T 8f416726a39948b4b634b696e4608443 P 1993589d7d8a4287930ff4d2bc25aab5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:08:33.127038 30137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:33.109280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50540 (local address 127.1.124.129:42445)
0808 13:08:33.109531 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:33.109540 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:33.113307 (+ 3767us) server_negotiation.cc:407] Connection header received
0808 13:08:33.113514 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:33.113524 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:33.113677 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:33.113811 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:33.117107 (+ 3296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:33.118825 (+ 1718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:33.121647 (+ 2822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:33.122628 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:33.122760 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:33.124726 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:33.124742 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:33.124766 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:33.125773 (+ 1007us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:33.125850 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:33.125900 (+ 50us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:33.126021 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:33.126333 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 13:08:33.126495 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 13:08:33.128968 29730 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:33.134809 30169 ts_tablet_manager.cc:1918] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:08:33.142267 30169 log.cc:1196] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/8f416726a39948b4b634b696e4608443
I20260808 13:08:33.142969 30171 ts_tablet_manager.cc:1918] T 8f416726a39948b4b634b696e4608443 P 1993589d7d8a4287930ff4d2bc25aab5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:08:33.143707 30171 log.cc:1196] T 8f416726a39948b4b634b696e4608443 P 1993589d7d8a4287930ff4d2bc25aab5: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/8f416726a39948b4b634b696e4608443
I20260808 13:08:33.144659 30169 ts_tablet_manager.cc:1939] T 8f416726a39948b4b634b696e4608443 P 401171b3bbcf40028b6f6c2f24c3c7f2: Deleting consensus metadata
I20260808 13:08:33.146816 30171 ts_tablet_manager.cc:1939] T 8f416726a39948b4b634b696e4608443 P 1993589d7d8a4287930ff4d2bc25aab5: Deleting consensus metadata
I20260808 13:08:33.146824 29435 catalog_manager.cc:4841] TS 401171b3bbcf40028b6f6c2f24c3c7f2 (127.1.124.130:41673): tablet 8f416726a39948b4b634b696e4608443 (table test-table [id=1d7ea092e9054e139d4156067c0df0e6]) successfully deleted
I20260808 13:08:33.148752 29437 catalog_manager.cc:4841] TS 1993589d7d8a4287930ff4d2bc25aab5 (127.1.124.131:43633): tablet 8f416726a39948b4b634b696e4608443 (table test-table [id=1d7ea092e9054e139d4156067c0df0e6]) successfully deleted
I20260808 13:08:33.241539 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 29540
I20260808 13:08:33.271849 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 29744
W20260808 13:08:33.271955 29437 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.124.129:42445 (error 108)
I20260808 13:08:33.274235 30136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:33.273706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:42445 (local address 127.0.0.1:50546)
0808 13:08:33.273882 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:33.274046 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:42445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 13:08:33.274643 29437 catalog_manager.cc:4584] TS 4d4dfb0356fc468cbe38e72982e72e09 (127.1.124.129:42445): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8f416726a39948b4b634b696e4608443: Network error: Client connection negotiation failed: client connection to 127.1.124.129:42445: connect: Connection refused (error 111)
I20260808 13:08:33.280543 30133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:33.273806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42300 (local address 127.1.124.130:41673)
0808 13:08:33.273980 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:33.273986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:33.274006 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:08:33.274068 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:33.274074 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:33.274178 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:33.274292 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:33.274843 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:33.275739 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:33.277244 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:33.277704 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:33.277867 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:33.279036 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:33.279048 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:33.279069 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:33.279700 (+ 631us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:33.279755 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:33.279775 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:33.279924 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:33.280097 (+ 173us) server_negotiation.cc:299] Negotiation successful
0808 13:08:33.280198 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 13:08:33.281186 29917 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:33.328269 30136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:33.327732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:42445 (local address 127.0.0.1:50552)
0808 13:08:33.327954 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:33.328040 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:42445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260808 13:08:33.329325 30136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:33.328978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:42445 (local address 127.0.0.1:50558)
0808 13:08:33.329126 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:33.329201 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:42445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260808 13:08:33.390358 30136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:33.389847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:42445 (local address 127.0.0.1:50574)
0808 13:08:33.390084 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:33.390185 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:42445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260808 13:08:33.391434 30136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:33.391098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:42445 (local address 127.0.0.1:50584)
0808 13:08:33.391255 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:33.391331 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:42445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260808 13:08:33.398694 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 29744
I20260808 13:08:33.421168 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 29929
I20260808 13:08:33.428300 30131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:33.422084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57100 (local address 127.1.124.131:43633)
0808 13:08:33.422296 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:33.422301 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:33.422320 (+ 19us) server_negotiation.cc:407] Connection header received
0808 13:08:33.422455 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:33.422459 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:33.422553 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:33.422643 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:33.423200 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:33.423948 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:33.425234 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:33.425624 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:33.425724 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:33.426779 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:33.426787 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:33.426803 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:33.427424 (+ 621us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:33.427472 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:33.427492 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:33.427584 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:33.427753 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 13:08:33.427845 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260808 13:08:33.429024 30102 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:33.472195 30136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:33.471566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:42445 (local address 127.0.0.1:50592)
0808 13:08:33.471809 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:33.471931 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:42445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260808 13:08:33.473402 30136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:33.472943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:42445 (local address 127.0.0.1:50606)
0808 13:08:33.473172 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:33.473262 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:42445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
I20260808 13:08:33.519169 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 29929
I20260808 13:08:33.540246 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 29405
I20260808 13:08:33.541395 29497 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:33.651894 30136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:33.651222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:42445 (local address 127.0.0.1:50622)
0808 13:08:33.651420 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:33.651580 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:42445: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 13:08:33.652199 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 29405
2026-08-08T13:08:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2993 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:08:33 dist-test-slave-t3q3 krb5kdc[30185](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:08:33 dist-test-slave-t3q3 krb5kdc[30185](info): set up 2 sockets
Aug 08 13:08:33 dist-test-slave-t3q3 krb5kdc[30185](info): commencing operation
krb5kdc: starting...
W20260808 13:08:35.012547 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.310s user 0.004s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:08:35 dist-test-slave-t3q3 krb5kdc[30185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:08:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:08:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:08:35.342819 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:35533
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:35533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:08:35.686385 30204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:35.686738 30204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:35.686900 30204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:35.694177 30204 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:08:35.694334 30204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:35.694448 30204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:35.694561 30204 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:08:35.694640 30204 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:08:35.704648 30204 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:35533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:35533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:35.706018 30204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:35.707369 30204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:35.720052 30210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:35.720170 30211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:35.720162 30213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:35.721310 30204 server_base.cc:1034] running on GCE node
I20260808 13:08:35.722208 30204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:35.723747 30204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:35.724961 30204 hybrid_clock.cc:648] HybridClock initialized: now 1786194515724913 us; error 45 us; skew 500 ppm
I20260808 13:08:35.725375 30204 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:08:35 dist-test-slave-t3q3 krb5kdc[30185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194515, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:08:35.732148 30204 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:08:35.733901 30204 webserver.cc:460] Webserver started at http://127.1.124.190:34511/ using document root <none> and password file <none>
I20260808 13:08:35.734653 30204 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:35.734797 30204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:35.735152 30204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:35.738288 30204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "b3e261adbc98483fbbf806e859ff146f"
format_stamp: "Formatted at 2026-08-08 13:08:35 on dist-test-slave-t3q3"
server_key: "0b5430e3ff4b2b2b92ab32a40665cb15"
server_key_iv: "c00dbe6929b576abc386462615cb90f7"
server_key_version: "encryptionkey@0"
I20260808 13:08:35.739077 30204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "b3e261adbc98483fbbf806e859ff146f"
format_stamp: "Formatted at 2026-08-08 13:08:35 on dist-test-slave-t3q3"
server_key: "0b5430e3ff4b2b2b92ab32a40665cb15"
server_key_iv: "c00dbe6929b576abc386462615cb90f7"
server_key_version: "encryptionkey@0"
I20260808 13:08:35.744591 30204 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260808 13:08:35.748191 30224 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:35.749648 30204 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:08:35.749863 30204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "b3e261adbc98483fbbf806e859ff146f"
format_stamp: "Formatted at 2026-08-08 13:08:35 on dist-test-slave-t3q3"
server_key: "0b5430e3ff4b2b2b92ab32a40665cb15"
server_key_iv: "c00dbe6929b576abc386462615cb90f7"
server_key_version: "encryptionkey@0"
I20260808 13:08:35.750125 30204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:35.767391 30204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:35.772015 30204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:35.772375 30204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:35.793277 30204 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:35533
I20260808 13:08:35.793323 30295 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:35533 every 8 connection(s)
I20260808 13:08:35.794710 30204 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:08:35.798487 30298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:35.803666 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 30204
I20260808 13:08:35.803875 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:08:35.804461 1522 external_mini_cluster.cc:1428] Setting key 217e1ac9d5610101b881188e2c4fe13f
I20260808 13:08:35.810343 30298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f: Bootstrap starting.
I20260808 13:08:35.814190 30298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:35.815502 30298 log.cc:824] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:35.818707 30298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f: No bootstrap required, opened a new log
Aug 08 13:08:35 dist-test-slave-t3q3 krb5kdc[30185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:08:35.827440 30298 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3e261adbc98483fbbf806e859ff146f" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 35533 } }
I20260808 13:08:35.827965 30298 raft_consensus.cc:374] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:35.828130 30298 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3e261adbc98483fbbf806e859ff146f, State: Initialized, Role: FOLLOWER
I20260808 13:08:35.828730 30298 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3e261adbc98483fbbf806e859ff146f" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 35533 } }
I20260808 13:08:35.828984 30298 raft_consensus.cc:388] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:08:35.829120 30302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:35.806382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52286 (local address 127.1.124.190:35533)
0808 13:08:35.807255 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:35.807267 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:35.807311 (+ 44us) server_negotiation.cc:407] Connection header received
0808 13:08:35.809161 (+ 1850us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:35.809185 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:35.809581 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:35.809904 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:35.811831 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:35.813315 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:35.815300 (+ 1985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:35.816255 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:35.816465 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:35.821055 (+ 4590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:35.821067 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:35.821083 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:35.821127 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:35.824501 (+ 3374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:35.825242 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:35.825250 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:35.825256 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:35.825426 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:35.825849 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:35.825854 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:35.825857 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:35.826466 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:35.826781 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:35.827286 (+ 505us) server_negotiation.cc:299] Negotiation successful
0808 13:08:35.827737 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":410,"threads_started":1}
I20260808 13:08:35.829154 30298 raft_consensus.cc:482] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:08:35.829371 30298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:35.831995 30298 raft_consensus.cc:504] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3e261adbc98483fbbf806e859ff146f" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 35533 } }
I20260808 13:08:35.832580 30298 leader_election.cc:302] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b3e261adbc98483fbbf806e859ff146f; no voters:
I20260808 13:08:35.833231 30298 leader_election.cc:288] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:08:35.833353 30305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:08:35.834120 30305 raft_consensus.cc:686] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f [term 1 LEADER]: Becoming Leader. State: Replica: b3e261adbc98483fbbf806e859ff146f, State: Running, Role: LEADER
I20260808 13:08:35.834893 30305 consensus_queue.cc:237] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3e261adbc98483fbbf806e859ff146f" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 35533 } }
I20260808 13:08:35.835563 30298 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:08:35.838161 30307 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f [sys.catalog]: SysCatalogTable state changed. Reason: New leader b3e261adbc98483fbbf806e859ff146f. Latest consensus state: current_term: 1 leader_uuid: "b3e261adbc98483fbbf806e859ff146f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3e261adbc98483fbbf806e859ff146f" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 35533 } } }
I20260808 13:08:35.838173 30306 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b3e261adbc98483fbbf806e859ff146f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b3e261adbc98483fbbf806e859ff146f" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 35533 } } }
I20260808 13:08:35.838987 30306 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f [sys.catalog]: This master's current role is: LEADER
I20260808 13:08:35.838995 30307 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f [sys.catalog]: This master's current role is: LEADER
I20260808 13:08:35.840898 30315 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:08:35.845346 30315 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:08:35.854121 30315 catalog_manager.cc:1347] Generated new cluster ID: 82828f43ae2c4eecb8e9a5293ecd6c78
I20260808 13:08:35.854297 30315 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:08:35.868530 30315 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:08:35.869813 30315 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:08:35.877243 30315 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f: Generated new TSK 0
I20260808 13:08:35.878058 30315 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.124.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
I20260808 13:08:35.955749 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:35533
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:08:36.302819 30336 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:36.303113 30336 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:36.303210 30336 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:36.310403 30336 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:36.310518 30336 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:36.310628 30336 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:08:36.321732 30336 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:35533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:36.322989 30336 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:36.324388 30336 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:36.338793 30344 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:36.338771 30345 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:36.338722 30347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:36.339737 30336 server_base.cc:1034] running on GCE node
I20260808 13:08:36.340466 30336 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:36.341408 30336 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:36.342650 30336 hybrid_clock.cc:648] HybridClock initialized: now 1786194516342610 us; error 63 us; skew 500 ppm
I20260808 13:08:36.342993 30336 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:08:36 dist-test-slave-t3q3 krb5kdc[30185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:08:36.346423 30336 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:08:36.347983 30336 webserver.cc:460] Webserver started at http://127.1.124.129:46091/ using document root <none> and password file <none>
I20260808 13:08:36.348729 30336 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:36.348886 30336 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:36.349231 30336 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:36.352200 30336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "92b39b3bf7e94a17bfc580ebc9b6972d"
format_stamp: "Formatted at 2026-08-08 13:08:36 on dist-test-slave-t3q3"
server_key: "930afc9dbfdcd19b2af02f9b88d4a886"
server_key_iv: "11d0da1a53f3ac5daac7dda9517c60e7"
server_key_version: "encryptionkey@0"
I20260808 13:08:36.353034 30336 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "92b39b3bf7e94a17bfc580ebc9b6972d"
format_stamp: "Formatted at 2026-08-08 13:08:36 on dist-test-slave-t3q3"
server_key: "930afc9dbfdcd19b2af02f9b88d4a886"
server_key_iv: "11d0da1a53f3ac5daac7dda9517c60e7"
server_key_version: "encryptionkey@0"
I20260808 13:08:36.358387 30336 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:08:36.361931 30358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:36.363435 30336 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:08:36.363631 30336 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "92b39b3bf7e94a17bfc580ebc9b6972d"
format_stamp: "Formatted at 2026-08-08 13:08:36 on dist-test-slave-t3q3"
server_key: "930afc9dbfdcd19b2af02f9b88d4a886"
server_key_iv: "11d0da1a53f3ac5daac7dda9517c60e7"
server_key_version: "encryptionkey@0"
I20260808 13:08:36.363835 30336 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:36.382879 30336 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:36.386829 30336 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:36.387140 30336 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:36.388130 30336 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:08:36.389424 30336 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:08:36.389532 30336 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:36.389685 30336 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:08:36.389760 30336 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:36.423236 30336 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:45757
I20260808 13:08:36.423281 30519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:45757 every 8 connection(s)
I20260808 13:08:36.424546 30336 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:08:36.426981 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 30336
I20260808 13:08:36.427176 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:08:36.427930 1522 external_mini_cluster.cc:1428] Setting key b920d6b795f6fbb100da05b1a2fe82ac
Aug 08 13:08:36 dist-test-slave-t3q3 krb5kdc[30185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:08:36.445701 30524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:36.427071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:41173 (local address 127.1.124.190:35533)
0808 13:08:36.429180 (+ 2109us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:36.429187 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:36.429212 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:08:36.430575 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:36.430580 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:36.430677 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:36.430805 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:36.432316 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:36.433067 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:36.434612 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:36.435018 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:36.435158 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:36.439226 (+ 4068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:36.439231 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:36.439235 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:36.439265 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:36.441944 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:36.442917 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:36.442924 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:36.442929 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:36.443088 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:36.443622 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:36.443629 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:36.443634 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:36.444000 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:36.444202 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:36.444906 (+ 704us) server_negotiation.cc:299] Negotiation successful
0808 13:08:36.445168 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1963,"thread_start_us":1837,"threads_started":1}
I20260808 13:08:36.446307 30528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:36.427575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:35533 (local address 127.1.124.129:41173)
0808 13:08:36.428428 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:36.428474 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:36.430311 (+ 1837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:36.431044 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:36.431052 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:36.431546 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:08:36.432146 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:36.432167 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:36.433241 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:36.433251 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:36.434448 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:36.434460 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:36.435257 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:36.435266 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:36.435490 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:36.436234 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:08:36.436261 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:08:36.438990 (+ 2729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:08:36.442096 (+ 3106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:08:36.442107 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:08:36.442126 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:08:36.442754 (+ 628us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:08:36.443218 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:08:36.443225 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:08:36.443230 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:08:36.443477 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:08:36.444163 (+ 686us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:08:36.444169 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:08:36.444508 (+ 339us) client_negotiation.cc:769] Sending connection context
0808 13:08:36.444736 (+ 228us) client_negotiation.cc:240] Negotiation successful
0808 13:08:36.445181 (+ 445us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":587,"thread_start_us":344,"threads_started":1}
I20260808 13:08:36.447896 30520 heartbeater.cc:347] Connected to a master server at 127.1.124.190:35533
I20260808 13:08:36.448256 30520 heartbeater.cc:464] Registering TS with master...
I20260808 13:08:36.448949 30520 heartbeater.cc:511] Master 127.1.124.190:35533 requested a full tablet report, sending...
I20260808 13:08:36.450922 30248 ts_manager.cc:194] Registered new tserver with Master: 92b39b3bf7e94a17bfc580ebc9b6972d (127.1.124.129:45757)
I20260808 13:08:36.452405 30248 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:41173
WARNING: no policy specified for kudu/127.1.124.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
I20260808 13:08:36.495468 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:35533
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:08:36.842761 30533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:36.843086 30533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:36.843258 30533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:36.850817 30533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:36.850970 30533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:36.851157 30533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:08:36.861207 30533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:35533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:36.862576 30533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:36.863878 30533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:36.877159 30540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:36.877334 30543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:36.877373 30541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:36.878861 30533 server_base.cc:1034] running on GCE node
I20260808 13:08:36.879496 30533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:36.880549 30533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:36.881788 30533 hybrid_clock.cc:648] HybridClock initialized: now 1786194516881745 us; error 38 us; skew 500 ppm
I20260808 13:08:36.882126 30533 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:08:36 dist-test-slave-t3q3 krb5kdc[30185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:08:36.885373 30533 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:08:36.886946 30533 webserver.cc:460] Webserver started at http://127.1.124.130:41925/ using document root <none> and password file <none>
I20260808 13:08:36.887629 30533 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:36.887785 30533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:36.888182 30533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:36.891040 30533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57"
format_stamp: "Formatted at 2026-08-08 13:08:36 on dist-test-slave-t3q3"
server_key: "91e3fbd6d1cbe67865721f5f5b8cb102"
server_key_iv: "5b2c9272dc7992b58e378f6ef9dd030f"
server_key_version: "encryptionkey@0"
I20260808 13:08:36.891878 30533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57"
format_stamp: "Formatted at 2026-08-08 13:08:36 on dist-test-slave-t3q3"
server_key: "91e3fbd6d1cbe67865721f5f5b8cb102"
server_key_iv: "5b2c9272dc7992b58e378f6ef9dd030f"
server_key_version: "encryptionkey@0"
I20260808 13:08:36.897135 30533 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 13:08:36.900605 30554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:36.901922 30533 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 13:08:36.902109 30533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57"
format_stamp: "Formatted at 2026-08-08 13:08:36 on dist-test-slave-t3q3"
server_key: "91e3fbd6d1cbe67865721f5f5b8cb102"
server_key_iv: "5b2c9272dc7992b58e378f6ef9dd030f"
server_key_version: "encryptionkey@0"
I20260808 13:08:36.902360 30533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:36.918807 30533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:36.922924 30533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:36.923235 30533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:36.924304 30533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:08:36.925639 30533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:08:36.925745 30533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:36.925900 30533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:08:36.925984 30533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:36.961221 30533 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:33135
I20260808 13:08:36.961270 30720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:33135 every 8 connection(s)
I20260808 13:08:36.962579 30533 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:08:36.964398 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 30533
I20260808 13:08:36.964612 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:08:36.965137 1522 external_mini_cluster.cc:1428] Setting key bbc9d1fcfbe1cc524f58357571a69b28
Aug 08 13:08:36 dist-test-slave-t3q3 krb5kdc[30185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194516, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:08:36.986444 30727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:36.965523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:60329 (local address 127.1.124.190:35533)
0808 13:08:36.966112 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:36.966119 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:36.967192 (+ 1073us) server_negotiation.cc:407] Connection header received
0808 13:08:36.969352 (+ 2160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:36.969361 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:36.969461 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:36.969594 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:36.971097 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:36.972312 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:36.973995 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:36.974754 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:36.974907 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:36.979127 (+ 4220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:36.979136 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:36.979143 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:36.979190 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:36.982524 (+ 3334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:36.983561 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:36.983569 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:36.983575 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:36.983765 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:36.984325 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:36.984333 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:36.984339 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:36.984746 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:36.984908 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:36.985662 (+ 754us) server_negotiation.cc:299] Negotiation successful
0808 13:08:36.985943 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":397,"thread_start_us":259,"threads_started":1}
I20260808 13:08:36.987146 30729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:36.966135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:35533 (local address 127.1.124.130:60329)
0808 13:08:36.966973 (+ 838us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:36.967011 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:36.969081 (+ 2070us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:36.969802 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:36.969810 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:36.970355 (+ 545us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:08:36.970914 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:36.970931 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:36.972473 (+ 1542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:36.972477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:36.973806 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:36.973820 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:36.975011 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:36.975016 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:36.975250 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:36.975981 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:08:36.976016 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:08:36.978860 (+ 2844us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:08:36.982687 (+ 3827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:08:36.982701 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:08:36.982720 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:08:36.983378 (+ 658us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:08:36.983922 (+ 544us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:08:36.983926 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:08:36.983929 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:08:36.984205 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:08:36.984904 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:08:36.984911 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:08:36.985264 (+ 353us) client_negotiation.cc:769] Sending connection context
0808 13:08:36.985506 (+ 242us) client_negotiation.cc:240] Negotiation successful
0808 13:08:36.985916 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":365,"threads_started":1}
I20260808 13:08:36.988842 30722 heartbeater.cc:347] Connected to a master server at 127.1.124.190:35533
I20260808 13:08:36.989158 30722 heartbeater.cc:464] Registering TS with master...
I20260808 13:08:36.989899 30722 heartbeater.cc:511] Master 127.1.124.190:35533 requested a full tablet report, sending...
I20260808 13:08:36.991447 30248 ts_manager.cc:194] Registered new tserver with Master: 6fc245e4aa0e4ab1b6ba4f4b92daaf57 (127.1.124.130:33135)
I20260808 13:08:36.992805 30248 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:60329
WARNING: no policy specified for kudu/127.1.124.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
I20260808 13:08:37.039276 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:35533
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:43413
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:08:37.389901 30738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:37.390199 30738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:37.390300 30738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:37.397456 30738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:37.397575 30738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:37.397691 30738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:08:37.408619 30738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:43413
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:35533
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:37.409958 30738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:37.411425 30738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:37.426609 30745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:37.426636 30746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:37.427109 30738 server_base.cc:1034] running on GCE node
W20260808 13:08:37.427336 30748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:37.428290 30738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:37.429356 30738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:37.430624 30738 hybrid_clock.cc:648] HybridClock initialized: now 1786194517430577 us; error 37 us; skew 500 ppm
I20260808 13:08:37.430977 30738 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:08:37 dist-test-slave-t3q3 krb5kdc[30185](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194517, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:08:37.434408 30738 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:08:37.435843 30738 webserver.cc:460] Webserver started at http://127.1.124.131:42663/ using document root <none> and password file <none>
I20260808 13:08:37.436621 30738 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:37.436728 30738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:37.437062 30738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:37.439973 30738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "cd5f3910c99646d689374da9ede04f36"
format_stamp: "Formatted at 2026-08-08 13:08:37 on dist-test-slave-t3q3"
server_key: "99be639d47881ddf1d46e3ffd41eefa3"
server_key_iv: "8c4274be128232c2546af4afe10de4f7"
server_key_version: "encryptionkey@0"
I20260808 13:08:37.440840 30738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "cd5f3910c99646d689374da9ede04f36"
format_stamp: "Formatted at 2026-08-08 13:08:37 on dist-test-slave-t3q3"
server_key: "99be639d47881ddf1d46e3ffd41eefa3"
server_key_iv: "8c4274be128232c2546af4afe10de4f7"
server_key_version: "encryptionkey@0"
I20260808 13:08:37.446208 30738 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:08:37.449891 30755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:37.451313 30738 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:08:37.451502 30738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "cd5f3910c99646d689374da9ede04f36"
format_stamp: "Formatted at 2026-08-08 13:08:37 on dist-test-slave-t3q3"
server_key: "99be639d47881ddf1d46e3ffd41eefa3"
server_key_iv: "8c4274be128232c2546af4afe10de4f7"
server_key_version: "encryptionkey@0"
I20260808 13:08:37.451719 30738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:37.455272 30520 heartbeater.cc:503] Master 127.1.124.190:35533 was elected leader, sending a full tablet report...
I20260808 13:08:37.482472 30738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:37.486903 30738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:37.487234 30738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:37.488302 30738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:08:37.489703 30738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:08:37.489779 30738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:37.489929 30738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:08:37.490015 30738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:37.525605 30738 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:37301
I20260808 13:08:37.525683 30917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:37301 every 8 connection(s)
I20260808 13:08:37.526986 30738 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:08:37.532804 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 30738
I20260808 13:08:37.533016 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:08:37.533504 1522 external_mini_cluster.cc:1428] Setting key b39449b76da237f5376cc9d5fe34c589
Aug 08 13:08:37 dist-test-slave-t3q3 krb5kdc[30185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194517, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:08:37.548981 30920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:37.529809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:52797 (local address 127.1.124.190:35533)
0808 13:08:37.530346 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:37.530350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:37.531161 (+ 811us) server_negotiation.cc:407] Connection header received
0808 13:08:37.533213 (+ 2052us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:37.533219 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:37.533332 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:37.533461 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:37.535352 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.536628 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.538383 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.538814 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.538923 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:37.542803 (+ 3880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:37.542809 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:37.542812 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:37.542846 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:37.545632 (+ 2786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:37.546394 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:37.546400 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:37.546404 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:37.546540 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:37.546993 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:37.547002 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:37.547004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:37.547282 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:37.547451 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:37.548384 (+ 933us) server_negotiation.cc:299] Negotiation successful
0808 13:08:37.548569 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":437,"thread_start_us":307,"threads_started":1}
I20260808 13:08:37.550205 30921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:37.530288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:35533 (local address 127.1.124.131:52797)
0808 13:08:37.530983 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:37.531029 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:37.532858 (+ 1829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:37.533727 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:37.533736 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:37.534407 (+ 671us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:08:37.535128 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:37.535147 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.536812 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.536817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:37.538226 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:37.538240 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.539041 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.539046 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:37.539269 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:37.540030 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:08:37.540054 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:08:37.542576 (+ 2522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:08:37.545787 (+ 3211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:08:37.545797 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:08:37.545815 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:08:37.546265 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:08:37.546656 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:08:37.546661 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:08:37.546665 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:08:37.546863 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:08:37.547416 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:08:37.547425 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:08:37.547879 (+ 454us) client_negotiation.cc:769] Sending connection context
0808 13:08:37.548251 (+ 372us) client_negotiation.cc:240] Negotiation successful
0808 13:08:37.548846 (+ 595us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":292,"threads_started":1}
I20260808 13:08:37.551628 30918 heartbeater.cc:347] Connected to a master server at 127.1.124.190:35533
I20260808 13:08:37.551942 30918 heartbeater.cc:464] Registering TS with master...
I20260808 13:08:37.552744 30918 heartbeater.cc:511] Master 127.1.124.190:35533 requested a full tablet report, sending...
I20260808 13:08:37.554248 30248 ts_manager.cc:194] Registered new tserver with Master: cd5f3910c99646d689374da9ede04f36 (127.1.124.131:37301)
I20260808 13:08:37.555212 30248 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:52797
I20260808 13:08:37.562130 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:08:37.580751 30920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:37.566440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52292 (local address 127.1.124.190:35533)
0808 13:08:37.566716 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:37.566723 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:37.567020 (+ 297us) server_negotiation.cc:407] Connection header received
0808 13:08:37.567263 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:37.567267 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:37.567343 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:37.567453 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:37.569098 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.570034 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.571351 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.571796 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.571902 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:37.574725 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:37.574730 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:37.574734 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:37.574766 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:37.577445 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:37.578260 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:37.578263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:37.578266 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:37.578385 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:37.578911 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:37.578915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:37.578917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:37.579440 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:37.579675 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:37.580179 (+ 504us) server_negotiation.cc:299] Negotiation successful
0808 13:08:37.580334 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 13:08:37.585093 30248 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52292:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:08:37.592126 30248 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:08:37.617427 30937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:37.604930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:37301 (local address 127.0.0.1:42070)
0808 13:08:37.605931 (+ 1001us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:37.605969 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:37.606225 (+ 256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:37.608955 (+ 2730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:37.608963 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:37.608990 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:37.610342 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:37.610360 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.612246 (+ 1886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.612251 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:37.614480 (+ 2229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:37.614496 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.616307 (+ 1811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.616322 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:37.616515 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:37.616565 (+ 50us) client_negotiation.cc:769] Sending connection context
0808 13:08:37.616710 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 13:08:37.616901 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":764,"mutex_wait_us":1,"thread_start_us":2194,"threads_started":1}
I20260808 13:08:37.618286 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:37.605832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59016 (local address 127.1.124.130:33135)
0808 13:08:37.606980 (+ 1148us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:37.606989 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:37.607690 (+ 701us) server_negotiation.cc:407] Connection header received
0808 13:08:37.607817 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:37.607827 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:37.608114 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:37.608320 (+ 206us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:37.610982 (+ 2662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.612424 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.615257 (+ 2833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.616649 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.616811 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:37.616984 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:37.617778 (+ 794us) server_negotiation.cc:299] Negotiation successful
0808 13:08:37.617956 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":996,"thread_start_us":872,"threads_started":1}
I20260808 13:08:37.618589 30939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:37.605228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42070 (local address 127.1.124.131:37301)
0808 13:08:37.608129 (+ 2901us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:37.608139 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:37.608171 (+ 32us) server_negotiation.cc:407] Connection header received
0808 13:08:37.608270 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:37.608277 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:37.608480 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:37.608687 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:37.610562 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.612103 (+ 1541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.614649 (+ 2546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.615880 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.616010 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:37.616298 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:37.616779 (+ 481us) server_negotiation.cc:299] Negotiation successful
0808 13:08:37.616947 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2712,"thread_start_us":2885,"threads_started":1}
I20260808 13:08:37.619021 30936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:37.605215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:33135 (local address 127.0.0.1:59016)
0808 13:08:37.607400 (+ 2185us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:37.607415 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:37.607574 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:37.608603 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:37.608613 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:37.608648 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:37.610823 (+ 2175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:37.610830 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.612591 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.612597 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:37.615075 (+ 2478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:37.615091 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.617150 (+ 2059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.617157 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:37.617350 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:37.617387 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 13:08:37.618576 (+ 1189us) client_negotiation.cc:240] Negotiation successful
0808 13:08:37.618738 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2077,"mutex_wait_us":523,"thread_start_us":394,"threads_started":1}
I20260808 13:08:37.619592 30941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:37.607623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:45757 (local address 127.0.0.1:37208)
0808 13:08:37.610608 (+ 2985us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:37.610626 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:37.610734 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:37.611305 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:37.611312 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:37.611337 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:37.611676 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:37.611688 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.613559 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.613564 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:37.615628 (+ 2064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:37.615639 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.618046 (+ 2407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.618051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:37.618167 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:37.618189 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 13:08:37.618251 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 13:08:37.618336 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2863,"mutex_wait_us":1,"thread_start_us":2685,"threads_started":1}
I20260808 13:08:37.620448 30944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:37.607830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37208 (local address 127.1.124.129:45757)
0808 13:08:37.609529 (+ 1699us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:37.609540 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:37.610746 (+ 1206us) server_negotiation.cc:407] Connection header received
0808 13:08:37.610856 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:37.610863 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:37.611101 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:37.611292 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:37.611869 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.613385 (+ 1516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.615784 (+ 2399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.617254 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.617406 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:37.619828 (+ 2422us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:37.619915 (+ 87us) server_negotiation.cc:299] Negotiation successful
0808 13:08:37.620141 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1528,"thread_start_us":1350,"threads_started":1}
I20260808 13:08:37.622707 30622 tablet_service.cc:1467] Processing CreateTablet for tablet 0aa854bcc80941d599b26b15c90eacce (DEFAULT_TABLE table=test-table [id=8a1ee130e5f64b8cbd73dc0c594f153e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:08:37.622599 30823 tablet_service.cc:1467] Processing CreateTablet for tablet 0aa854bcc80941d599b26b15c90eacce (DEFAULT_TABLE table=test-table [id=8a1ee130e5f64b8cbd73dc0c594f153e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:08:37.624248 30823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aa854bcc80941d599b26b15c90eacce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:37.624260 30622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aa854bcc80941d599b26b15c90eacce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:37.623845 30426 tablet_service.cc:1467] Processing CreateTablet for tablet 0aa854bcc80941d599b26b15c90eacce (DEFAULT_TABLE table=test-table [id=8a1ee130e5f64b8cbd73dc0c594f153e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:08:37.625172 30426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0aa854bcc80941d599b26b15c90eacce. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:37.642385 30946 tablet_bootstrap.cc:492] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57: Bootstrap starting.
I20260808 13:08:37.646127 30948 tablet_bootstrap.cc:492] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d: Bootstrap starting.
I20260808 13:08:37.649168 30946 tablet_bootstrap.cc:654] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:37.650774 30946 log.cc:824] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:37.653074 30945 tablet_bootstrap.cc:492] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36: Bootstrap starting.
I20260808 13:08:37.655625 30948 tablet_bootstrap.cc:654] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:37.655843 30946 tablet_bootstrap.cc:492] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57: No bootstrap required, opened a new log
I20260808 13:08:37.656313 30946 ts_tablet_manager.cc:1397] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260808 13:08:37.657362 30948 log.cc:824] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:37.659025 30945 tablet_bootstrap.cc:654] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:37.660431 30945 log.cc:824] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:37.660827 30948 tablet_bootstrap.cc:492] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d: No bootstrap required, opened a new log
I20260808 13:08:37.661274 30948 ts_tablet_manager.cc:1397] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d: Time spent bootstrapping tablet: real 0.015s user 0.009s sys 0.000s
I20260808 13:08:37.666759 30945 tablet_bootstrap.cc:492] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36: No bootstrap required, opened a new log
I20260808 13:08:37.667174 30945 ts_tablet_manager.cc:1397] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.003s
I20260808 13:08:37.668531 30946 raft_consensus.cc:348] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5f3910c99646d689374da9ede04f36" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37301 } } peers { permanent_uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33135 } } peers { permanent_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45757 } }
I20260808 13:08:37.669081 30946 raft_consensus.cc:374] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:37.669219 30946 raft_consensus.cc:729] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fc245e4aa0e4ab1b6ba4f4b92daaf57, State: Initialized, Role: FOLLOWER
I20260808 13:08:37.669920 30946 consensus_queue.cc:260] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5f3910c99646d689374da9ede04f36" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37301 } } peers { permanent_uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33135 } } peers { permanent_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45757 } }
I20260808 13:08:37.671167 30722 heartbeater.cc:503] Master 127.1.124.190:35533 was elected leader, sending a full tablet report...
I20260808 13:08:37.671953 30946 ts_tablet_manager.cc:1428] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57: Time spent starting tablet: real 0.015s user 0.010s sys 0.005s
I20260808 13:08:37.674036 30948 raft_consensus.cc:348] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5f3910c99646d689374da9ede04f36" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37301 } } peers { permanent_uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33135 } } peers { permanent_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45757 } }
I20260808 13:08:37.674518 30948 raft_consensus.cc:374] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:37.674642 30948 raft_consensus.cc:729] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92b39b3bf7e94a17bfc580ebc9b6972d, State: Initialized, Role: FOLLOWER
I20260808 13:08:37.675333 30948 consensus_queue.cc:260] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5f3910c99646d689374da9ede04f36" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37301 } } peers { permanent_uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33135 } } peers { permanent_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45757 } }
I20260808 13:08:37.678251 30948 ts_tablet_manager.cc:1428] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d: Time spent starting tablet: real 0.017s user 0.011s sys 0.004s
I20260808 13:08:37.678608 30945 raft_consensus.cc:348] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5f3910c99646d689374da9ede04f36" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37301 } } peers { permanent_uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33135 } } peers { permanent_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45757 } }
I20260808 13:08:37.679143 30945 raft_consensus.cc:374] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:37.679252 30945 raft_consensus.cc:729] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd5f3910c99646d689374da9ede04f36, State: Initialized, Role: FOLLOWER
I20260808 13:08:37.679935 30945 consensus_queue.cc:260] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5f3910c99646d689374da9ede04f36" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37301 } } peers { permanent_uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33135 } } peers { permanent_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45757 } }
I20260808 13:08:37.681908 30918 heartbeater.cc:503] Master 127.1.124.190:35533 was elected leader, sending a full tablet report...
I20260808 13:08:37.682637 30945 ts_tablet_manager.cc:1428] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36: Time spent starting tablet: real 0.015s user 0.012s sys 0.000s
W20260808 13:08:37.714764 30724 tablet.cc:2369] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:08:37.772179 30955 raft_consensus.cc:482] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:08:37.772518 30955 raft_consensus.cc:504] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5f3910c99646d689374da9ede04f36" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37301 } } peers { permanent_uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33135 } } peers { permanent_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45757 } }
I20260808 13:08:37.774430 30955 leader_election.cc:288] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd5f3910c99646d689374da9ede04f36 (127.1.124.131:37301), 6fc245e4aa0e4ab1b6ba4f4b92daaf57 (127.1.124.130:33135)
I20260808 13:08:37.778671 30954 raft_consensus.cc:482] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
W20260808 13:08:37.778797 30919 tablet.cc:2369] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:08:37.778981 30954 raft_consensus.cc:504] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5f3910c99646d689374da9ede04f36" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37301 } } peers { permanent_uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33135 } } peers { permanent_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45757 } }
I20260808 13:08:37.781158 30954 leader_election.cc:288] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd5f3910c99646d689374da9ede04f36 (127.1.124.131:37301), 92b39b3bf7e94a17bfc580ebc9b6972d (127.1.124.129:45757)
I20260808 13:08:37.784301 30959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:37.774998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:33135 (local address 127.1.124.129:42135)
0808 13:08:37.775941 (+ 943us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:37.775956 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:37.776131 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:37.776579 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:37.776583 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:37.776611 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:37.777045 (+ 434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:37.777057 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.778198 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.778203 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:37.780633 (+ 2430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:37.780647 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.782955 (+ 2308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.782961 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:37.783529 (+ 568us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:37.783559 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 13:08:37.783651 (+ 92us) client_negotiation.cc:240] Negotiation successful
0808 13:08:37.783802 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":840,"mutex_wait_us":396,"thread_start_us":569,"threads_started":1}
I20260808 13:08:37.784338 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:37.775033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:42135 (local address 127.1.124.130:33135)
0808 13:08:37.775300 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:37.775309 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:37.776030 (+ 721us) server_negotiation.cc:407] Connection header received
0808 13:08:37.776332 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:37.776336 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:37.776428 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:37.776527 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:37.777253 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.778025 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.780804 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.782231 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.782366 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:37.782439 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:37.783703 (+ 1264us) server_negotiation.cc:299] Negotiation successful
0808 13:08:37.783956 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260808 13:08:37.784984 30939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:37.774879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:43069 (local address 127.1.124.131:37301)
0808 13:08:37.775098 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:37.775105 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:37.775946 (+ 841us) server_negotiation.cc:407] Connection header received
0808 13:08:37.776210 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:37.776219 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:37.776305 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:37.776396 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:37.777017 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.777879 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.780913 (+ 3034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.782193 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.782379 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:37.782444 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:37.784552 (+ 2108us) server_negotiation.cc:299] Negotiation successful
0808 13:08:37.784709 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 13:08:37.785701 30958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:37.774613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:37301 (local address 127.1.124.129:43069)
0808 13:08:37.775817 (+ 1204us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:37.775842 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:37.776009 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:37.776434 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:37.776438 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:37.776465 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:37.776857 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:37.776867 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.778024 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.778030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:37.780652 (+ 2622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:37.780667 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.782640 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.782648 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:37.783213 (+ 565us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:37.783261 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 13:08:37.785164 (+ 1903us) client_negotiation.cc:240] Negotiation successful
0808 13:08:37.785304 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1017,"mutex_wait_us":125,"thread_start_us":692,"threads_started":1}
I20260808 13:08:37.785817 30651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aa854bcc80941d599b26b15c90eacce" candidate_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57" is_pre_election: true
I20260808 13:08:37.786631 30853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aa854bcc80941d599b26b15c90eacce" candidate_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd5f3910c99646d689374da9ede04f36" is_pre_election: true
I20260808 13:08:37.786983 30651 raft_consensus.cc:2454] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92b39b3bf7e94a17bfc580ebc9b6972d in term 0.
I20260808 13:08:37.787216 30853 raft_consensus.cc:2454] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92b39b3bf7e94a17bfc580ebc9b6972d in term 0.
I20260808 13:08:37.787895 30360 leader_election.cc:302] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6fc245e4aa0e4ab1b6ba4f4b92daaf57, 92b39b3bf7e94a17bfc580ebc9b6972d, cd5f3910c99646d689374da9ede04f36; no voters:
I20260808 13:08:37.788388 30955 raft_consensus.cc:2781] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:08:37.788556 30955 raft_consensus.cc:482] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:08:37.788698 30955 raft_consensus.cc:3037] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:37.790778 30944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:37.782629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:49949 (local address 127.1.124.129:45757)
0808 13:08:37.782810 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:37.782816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:37.783293 (+ 477us) server_negotiation.cc:407] Connection header received
0808 13:08:37.783547 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:37.783552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:37.783643 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:37.783723 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:37.784524 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.785555 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.788895 (+ 3340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.789741 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.789966 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:37.790042 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:37.790395 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 13:08:37.790544 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 13:08:37.790923 30963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:37.782280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:45757 (local address 127.1.124.130:49949)
0808 13:08:37.783195 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:37.783216 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:37.783407 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:37.783863 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:37.783868 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:37.783894 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:37.784333 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:37.784347 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.785776 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.785928 (+ 152us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:37.788722 (+ 2794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:37.788737 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.789988 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.789994 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:37.790172 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:37.790216 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 13:08:37.790361 (+ 145us) client_negotiation.cc:240] Negotiation successful
0808 13:08:37.790500 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"mutex_wait_us":5,"thread_start_us":589,"threads_started":1}
I20260808 13:08:37.791167 30960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:37.781269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:37301 (local address 127.1.124.130:57797)
0808 13:08:37.781994 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:37.782018 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:37.782164 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:37.783193 (+ 1029us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:37.783198 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:37.783241 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:37.783751 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:37.783762 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.785691 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.785696 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:37.788511 (+ 2815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:37.788533 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.790258 (+ 1725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.790267 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:37.790434 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:37.790463 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 13:08:37.790541 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 13:08:37.790675 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"spinlock_wait_cycles":24960,"thread_start_us":622,"threads_started":1}
I20260808 13:08:37.791869 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:37.781604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:57797 (local address 127.1.124.131:37301)
0808 13:08:37.782868 (+ 1264us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:37.782876 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:37.782899 (+ 23us) server_negotiation.cc:407] Connection header received
0808 13:08:37.782961 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:37.782966 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:37.783040 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:37.783147 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:37.784080 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.785506 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.788731 (+ 3225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.790047 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.790226 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:37.790296 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:37.791372 (+ 1076us) server_negotiation.cc:299] Negotiation successful
0808 13:08:37.791497 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1127,"thread_start_us":973,"threads_started":1}
I20260808 13:08:37.792002 30453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aa854bcc80941d599b26b15c90eacce" candidate_uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d" is_pre_election: true
I20260808 13:08:37.792296 30955 raft_consensus.cc:504] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5f3910c99646d689374da9ede04f36" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37301 } } peers { permanent_uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33135 } } peers { permanent_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45757 } }
I20260808 13:08:37.796387 30853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aa854bcc80941d599b26b15c90eacce" candidate_uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd5f3910c99646d689374da9ede04f36" is_pre_election: true
I20260808 13:08:37.796625 30853 raft_consensus.cc:2454] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fc245e4aa0e4ab1b6ba4f4b92daaf57 in term 0.
I20260808 13:08:37.796882 30453 raft_consensus.cc:2379] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6fc245e4aa0e4ab1b6ba4f4b92daaf57 in current term 1: Already voted for candidate 92b39b3bf7e94a17bfc580ebc9b6972d in this term.
I20260808 13:08:37.797318 30559 leader_election.cc:302] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fc245e4aa0e4ab1b6ba4f4b92daaf57, cd5f3910c99646d689374da9ede04f36; no voters:
I20260808 13:08:37.797638 30853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aa854bcc80941d599b26b15c90eacce" candidate_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd5f3910c99646d689374da9ede04f36"
I20260808 13:08:37.797741 30651 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aa854bcc80941d599b26b15c90eacce" candidate_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57"
I20260808 13:08:37.797843 30954 raft_consensus.cc:2781] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:08:37.797974 30651 raft_consensus.cc:3037] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:37.797902 30853 raft_consensus.cc:3037] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:37.798422 30955 leader_election.cc:288] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [CANDIDATE]: Term 1 election: Requested vote from peers cd5f3910c99646d689374da9ede04f36 (127.1.124.131:37301), 6fc245e4aa0e4ab1b6ba4f4b92daaf57 (127.1.124.130:33135)
I20260808 13:08:37.801010 30651 raft_consensus.cc:2454] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92b39b3bf7e94a17bfc580ebc9b6972d in term 1.
I20260808 13:08:37.801186 30954 raft_consensus.cc:482] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:08:37.801321 30954 raft_consensus.cc:3037] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:08:37.801540 30853 raft_consensus.cc:2454] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92b39b3bf7e94a17bfc580ebc9b6972d in term 1.
I20260808 13:08:37.802067 30362 leader_election.cc:302] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92b39b3bf7e94a17bfc580ebc9b6972d, cd5f3910c99646d689374da9ede04f36; no voters:
I20260808 13:08:37.802476 30955 raft_consensus.cc:2781] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:08:37.803025 30955 raft_consensus.cc:686] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [term 1 LEADER]: Becoming Leader. State: Replica: 92b39b3bf7e94a17bfc580ebc9b6972d, State: Running, Role: LEADER
I20260808 13:08:37.803665 30955 consensus_queue.cc:237] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5f3910c99646d689374da9ede04f36" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37301 } } peers { permanent_uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33135 } } peers { permanent_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45757 } }
I20260808 13:08:37.804427 30954 raft_consensus.cc:504] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5f3910c99646d689374da9ede04f36" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37301 } } peers { permanent_uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33135 } } peers { permanent_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45757 } }
I20260808 13:08:37.805975 30954 leader_election.cc:288] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [CANDIDATE]: Term 2 election: Requested vote from peers cd5f3910c99646d689374da9ede04f36 (127.1.124.131:37301), 92b39b3bf7e94a17bfc580ebc9b6972d (127.1.124.129:45757)
I20260808 13:08:37.806525 30853 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aa854bcc80941d599b26b15c90eacce" candidate_uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd5f3910c99646d689374da9ede04f36"
I20260808 13:08:37.806711 30453 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0aa854bcc80941d599b26b15c90eacce" candidate_uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d"
I20260808 13:08:37.806999 30853 raft_consensus.cc:3037] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36 [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:08:37.810003 30853 raft_consensus.cc:2454] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fc245e4aa0e4ab1b6ba4f4b92daaf57 in term 2.
I20260808 13:08:37.810457 30559 leader_election.cc:302] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fc245e4aa0e4ab1b6ba4f4b92daaf57, cd5f3910c99646d689374da9ede04f36; no voters: 92b39b3bf7e94a17bfc580ebc9b6972d
I20260808 13:08:37.810926 30954 raft_consensus.cc:2781] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 13:08:37.812178 30954 raft_consensus.cc:686] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [term 2 LEADER]: Becoming Leader. State: Replica: 6fc245e4aa0e4ab1b6ba4f4b92daaf57, State: Running, Role: LEADER
I20260808 13:08:37.812674 30954 consensus_queue.cc:237] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5f3910c99646d689374da9ede04f36" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37301 } } peers { permanent_uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33135 } } peers { permanent_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45757 } }
I20260808 13:08:37.812522 30248 catalog_manager.cc:5495] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d reported cstate change: term changed from 0 to 1, leader changed from <none> to 92b39b3bf7e94a17bfc580ebc9b6972d (127.1.124.129). New cstate: current_term: 1 leader_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5f3910c99646d689374da9ede04f36" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45757 } health_report { overall_health: HEALTHY } } }
I20260808 13:08:37.817323 30248 catalog_manager.cc:5495] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 reported cstate change: term changed from 1 to 2, leader changed from 92b39b3bf7e94a17bfc580ebc9b6972d (127.1.124.129) to 6fc245e4aa0e4ab1b6ba4f4b92daaf57 (127.1.124.130). New cstate: current_term: 2 leader_uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5f3910c99646d689374da9ede04f36" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33135 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45757 } health_report { overall_health: UNKNOWN } } }
I20260808 13:08:37.868253 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:37.862231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59018 (local address 127.1.124.130:33135)
0808 13:08:37.862402 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:37.862407 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:37.862427 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:08:37.862522 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:37.862525 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:37.862601 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:37.862745 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:08:37.863128 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.863950 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.865675 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.866110 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.866222 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:37.866790 (+ 568us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:08:37.867404 (+ 614us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:08:37.867573 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:37.867848 (+ 275us) server_negotiation.cc:299] Negotiation successful
0808 13:08:37.867905 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 13:08:37.876293 30453 raft_consensus.cc:3032] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [term 1 LEADER]: Stepping down as leader of term 1
I20260808 13:08:37.876482 30453 raft_consensus.cc:729] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 92b39b3bf7e94a17bfc580ebc9b6972d, State: Running, Role: LEADER
I20260808 13:08:37.876631 30853 raft_consensus.cc:1261] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36 [term 2 FOLLOWER]: Refusing update from remote peer 6fc245e4aa0e4ab1b6ba4f4b92daaf57: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260808 13:08:37.876857 30453 consensus_queue.cc:260] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cd5f3910c99646d689374da9ede04f36" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37301 } } peers { permanent_uuid: "6fc245e4aa0e4ab1b6ba4f4b92daaf57" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33135 } } peers { permanent_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45757 } }
I20260808 13:08:37.877358 30453 raft_consensus.cc:3037] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [term 1 FOLLOWER]: Advancing to term 2
I20260808 13:08:37.877555 30967 consensus_queue.cc:1035] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd5f3910c99646d689374da9ede04f36" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 37301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:08:37.880211 30453 raft_consensus.cc:1261] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [term 2 FOLLOWER]: Refusing update from remote peer 6fc245e4aa0e4ab1b6ba4f4b92daaf57: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260808 13:08:37.880913 30967 consensus_queue.cc:1035] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92b39b3bf7e94a17bfc580ebc9b6972d" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 45757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:08:37.881155 30967 consensus_queue.cc:1225] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [LEADER]: Peer 92b39b3bf7e94a17bfc580ebc9b6972d log is divergent from this leader: its last log entry 1.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260808 13:08:37.882776 30453 pending_rounds.cc:76] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d: Aborting all ops after (but not including) 0
I20260808 13:08:37.882958 30453 pending_rounds.cc:97] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260808 13:08:37.883147 30453 raft_consensus.cc:2866] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260808 13:08:37.902509 30246 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:08:37.902874 30246 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52292:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:08:37.907186 30246 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b3e261adbc98483fbbf806e859ff146f: Sending DeleteTablet for 3 replicas of tablet 0aa854bcc80941d599b26b15c90eacce
I20260808 13:08:37.908424 30622 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aa854bcc80941d599b26b15c90eacce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:08:37 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:59016
I20260808 13:08:37.908617 30426 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aa854bcc80941d599b26b15c90eacce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:08:37 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:37208
I20260808 13:08:37.909452 30979 tablet_replica.cc:330] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57: stopping tablet replica
I20260808 13:08:37.909710 30823 tablet_service.cc:1514] Processing DeleteTablet for tablet 0aa854bcc80941d599b26b15c90eacce with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:08:37 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:42070
I20260808 13:08:37.911026 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 30336
I20260808 13:08:37.919525 30981 tablet_replica.cc:330] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d: stopping tablet replica
I20260808 13:08:37.920383 30981 raft_consensus.cc:2229] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [term 2 FOLLOWER]: Raft consensus shutting down.
I20260808 13:08:37.920902 30979 raft_consensus.cc:2229] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [term 2 LEADER]: Raft consensus shutting down.
I20260808 13:08:37.921069 30981 raft_consensus.cc:2258] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d [term 2 FOLLOWER]: Raft consensus is shut down!
I20260808 13:08:37.921907 30979 raft_consensus.cc:2258] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260808 13:08:37.921988 30982 tablet_replica.cc:330] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36: stopping tablet replica
I20260808 13:08:37.922418 30982 raft_consensus.cc:2229] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260808 13:08:37.923065 30982 raft_consensus.cc:2258] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260808 13:08:37.924324 30981 ts_tablet_manager.cc:1905] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:08:37.924481 30979 ts_tablet_manager.cc:1905] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:08:37.925189 30982 ts_tablet_manager.cc:1905] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:08:37.931489 30981 ts_tablet_manager.cc:1918] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260808 13:08:37.932129 30982 ts_tablet_manager.cc:1918] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260808 13:08:37.932143 30979 ts_tablet_manager.cc:1918] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260808 13:08:37.932337 30979 log.cc:1196] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/0aa854bcc80941d599b26b15c90eacce
I20260808 13:08:37.932396 30982 log.cc:1196] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/0aa854bcc80941d599b26b15c90eacce
I20260808 13:08:37.933037 30979 ts_tablet_manager.cc:1939] T 0aa854bcc80941d599b26b15c90eacce P 6fc245e4aa0e4ab1b6ba4f4b92daaf57: Deleting consensus metadata
I20260808 13:08:37.933116 30982 ts_tablet_manager.cc:1939] T 0aa854bcc80941d599b26b15c90eacce P cd5f3910c99646d689374da9ede04f36: Deleting consensus metadata
I20260808 13:08:37.935105 30228 catalog_manager.cc:4841] TS 6fc245e4aa0e4ab1b6ba4f4b92daaf57 (127.1.124.130:33135): tablet 0aa854bcc80941d599b26b15c90eacce (table test-table [id=8a1ee130e5f64b8cbd73dc0c594f153e]) successfully deleted
I20260808 13:08:37.936023 30231 catalog_manager.cc:4841] TS cd5f3910c99646d689374da9ede04f36 (127.1.124.131:37301): tablet 0aa854bcc80941d599b26b15c90eacce (table test-table [id=8a1ee130e5f64b8cbd73dc0c594f153e]) successfully deleted
I20260808 13:08:37.936467 30981 log.cc:1196] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/0aa854bcc80941d599b26b15c90eacce
I20260808 13:08:37.937083 30981 ts_tablet_manager.cc:1939] T 0aa854bcc80941d599b26b15c90eacce P 92b39b3bf7e94a17bfc580ebc9b6972d: Deleting consensus metadata
I20260808 13:08:37.938755 30231 catalog_manager.cc:4841] TS 92b39b3bf7e94a17bfc580ebc9b6972d (127.1.124.129:45757): tablet 0aa854bcc80941d599b26b15c90eacce (table test-table [id=8a1ee130e5f64b8cbd73dc0c594f153e]) successfully deleted
Aug 08 13:08:37 dist-test-slave-t3q3 krb5kdc[30185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.129@KRBTEST.COM
I20260808 13:08:37.951659 30944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:37.913873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37212 (local address 127.1.124.129:45757)
0808 13:08:37.914058 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:37.914066 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:37.922025 (+ 7959us) server_negotiation.cc:407] Connection header received
0808 13:08:37.932020 (+ 9995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:37.932026 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:37.932162 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:37.932262 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:37.934297 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.936764 (+ 2467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.938755 (+ 1991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:37.939392 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:37.939525 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:37.944112 (+ 4587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:37.944128 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:37.944152 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:37.944194 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:37.948089 (+ 3895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:37.948879 (+ 790us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:37.948889 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:37.948893 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:37.949086 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:37.949634 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:37.949642 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:37.949645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:37.950144 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:37.950498 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:37.950985 (+ 487us) server_negotiation.cc:299] Negotiation successful
0808 13:08:37.951259 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 13:08:37.952512 30514 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:38.077343 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 30336
I20260808 13:08:38.108942 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 30533
Aug 08 13:08:38 dist-test-slave-t3q3 krb5kdc[30185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.130@KRBTEST.COM
I20260808 13:08:38.127616 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:38.109816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59024 (local address 127.1.124.130:33135)
0808 13:08:38.110023 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:38.110027 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:38.110164 (+ 137us) server_negotiation.cc:407] Connection header received
0808 13:08:38.110349 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:38.110353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:38.110453 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:38.110576 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:38.112398 (+ 1822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:38.113191 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:38.115119 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:38.115629 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:38.115727 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:38.121278 (+ 5551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:38.121292 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:38.121307 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:38.121348 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:38.124660 (+ 3312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:38.125420 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:38.125428 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:38.125431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:38.125581 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:38.125972 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:38.125977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:38.125981 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:38.126388 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:38.126687 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:38.127021 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 13:08:38.127262 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 13:08:38.128363 30715 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:38.246265 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 30533
I20260808 13:08:38.274968 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 30738
Aug 08 13:08:38 dist-test-slave-t3q3 krb5kdc[30185](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194515, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.131@KRBTEST.COM
I20260808 13:08:38.290983 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:38.275766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42086 (local address 127.1.124.131:37301)
0808 13:08:38.275966 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:38.275971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:38.275993 (+ 22us) server_negotiation.cc:407] Connection header received
0808 13:08:38.276253 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:38.276258 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:38.276325 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:38.276428 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:38.277784 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:38.278536 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:38.279860 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:38.280328 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:38.280427 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:38.285184 (+ 4757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:38.285228 (+ 44us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:38.285251 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:38.285293 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:38.288279 (+ 2986us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:38.289030 (+ 751us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:38.289039 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:38.289041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:38.289181 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:38.289583 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:38.289586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:38.289588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:38.289935 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:38.290171 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:38.290543 (+ 372us) server_negotiation.cc:299] Negotiation successful
0808 13:08:38.290738 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260808 13:08:38.291680 30908 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:38.385541 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 30738
I20260808 13:08:38.412177 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 30204
I20260808 13:08:38.413422 30290 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:38.525146 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 30204
2026-08-08T13:08:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4872 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T13:08:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:08:38Z Disabled control of system clock
I20260808 13:08:38.620755 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:42829
--webserver_interface=127.1.124.190
--webserver_port=0
--builtin_ntp_servers=127.1.124.148:42321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:42829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 13:08:38.972625 30994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:38.972926 30994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:38.973035 30994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:38.980425 30994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:08:38.980542 30994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:38.980582 30994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:38.980635 30994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:08:38.980680 30994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:08:38.990061 30994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:42321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:42829
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:42829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:38.991380 30994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:38.993309 30994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:39.006426 30999 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:39.006330 31002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:39.006359 31005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:39.007530 30994 server_base.cc:1034] running on GCE node
I20260808 13:08:39.008531 30994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:39.009972 30994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:39.011174 30994 hybrid_clock.cc:648] HybridClock initialized: now 1786194519011108 us; error 60 us; skew 500 ppm
I20260808 13:08:39.011552 30994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:08:39.014041 30994 webserver.cc:460] Webserver started at http://127.1.124.190:45993/ using document root <none> and password file <none>
I20260808 13:08:39.014791 30994 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:39.014886 30994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:39.015272 30994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:39.018930 30994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "837731b5fcd54994a4b28e2336fd87f3"
format_stamp: "Formatted at 2026-08-08 13:08:39 on dist-test-slave-t3q3"
server_key: "08d46763b4573d7753a85436d770e106"
server_key_iv: "205e07cccff31f10157e73faf87fdf50"
server_key_version: "encryptionkey@0"
I20260808 13:08:39.019797 30994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "837731b5fcd54994a4b28e2336fd87f3"
format_stamp: "Formatted at 2026-08-08 13:08:39 on dist-test-slave-t3q3"
server_key: "08d46763b4573d7753a85436d770e106"
server_key_iv: "205e07cccff31f10157e73faf87fdf50"
server_key_version: "encryptionkey@0"
I20260808 13:08:39.025256 30994 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 13:08:39.028872 31013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:39.030329 30994 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:08:39.030532 30994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "837731b5fcd54994a4b28e2336fd87f3"
format_stamp: "Formatted at 2026-08-08 13:08:39 on dist-test-slave-t3q3"
server_key: "08d46763b4573d7753a85436d770e106"
server_key_iv: "205e07cccff31f10157e73faf87fdf50"
server_key_version: "encryptionkey@0"
I20260808 13:08:39.030764 30994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:39.064296 30994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:39.065227 30994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:39.065541 30994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:39.086184 30994 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:42829
I20260808 13:08:39.086206 31089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:42829 every 8 connection(s)
I20260808 13:08:39.087664 30994 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:08:39.089368 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 30994
I20260808 13:08:39.089599 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:08:39.090075 1522 external_mini_cluster.cc:1428] Setting key 22fe4d499e7d175d79827e1cfd5acb2c
I20260808 13:08:39.092494 31091 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:39.104261 31094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:39.092133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32988 (local address 127.1.124.190:42829)
0808 13:08:39.093277 (+ 1144us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:39.093289 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:39.093332 (+ 43us) server_negotiation.cc:407] Connection header received
0808 13:08:39.094986 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:39.095010 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:39.095506 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:39.095941 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:39.096556 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:39.097951 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:39.099360 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:39.100119 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:39.100333 (+ 214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:39.101646 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:39.101655 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:39.101670 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:39.102309 (+ 639us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:39.102408 (+ 99us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:39.102431 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:39.102565 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:39.102843 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 13:08:39.103194 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":674,"thread_start_us":439,"threads_started":1}
I20260808 13:08:39.104784 31091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3: Bootstrap starting.
I20260808 13:08:39.108474 31091 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:39.109673 31091 log.cc:824] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:39.113076 31091 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3: No bootstrap required, opened a new log
I20260808 13:08:39.121644 31091 raft_consensus.cc:348] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "837731b5fcd54994a4b28e2336fd87f3" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42829 } }
I20260808 13:08:39.122073 31091 raft_consensus.cc:374] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:39.122216 31091 raft_consensus.cc:729] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 837731b5fcd54994a4b28e2336fd87f3, State: Initialized, Role: FOLLOWER
I20260808 13:08:39.122835 31091 consensus_queue.cc:260] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "837731b5fcd54994a4b28e2336fd87f3" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42829 } }
I20260808 13:08:39.123100 31091 raft_consensus.cc:388] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:08:39.123253 31091 raft_consensus.cc:482] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:08:39.123462 31091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:39.126237 31091 raft_consensus.cc:504] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "837731b5fcd54994a4b28e2336fd87f3" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42829 } }
I20260808 13:08:39.126912 31091 leader_election.cc:302] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 837731b5fcd54994a4b28e2336fd87f3; no voters:
I20260808 13:08:39.132498 31091 leader_election.cc:288] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:08:39.132623 31096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:08:39.135502 31091 sys_catalog.cc:564] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:08:39.136394 31096 raft_consensus.cc:686] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3 [term 1 LEADER]: Becoming Leader. State: Replica: 837731b5fcd54994a4b28e2336fd87f3, State: Running, Role: LEADER
I20260808 13:08:39.137045 31096 consensus_queue.cc:237] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "837731b5fcd54994a4b28e2336fd87f3" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42829 } }
I20260808 13:08:39.140559 31100 sys_catalog.cc:455] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 837731b5fcd54994a4b28e2336fd87f3. Latest consensus state: current_term: 1 leader_uuid: "837731b5fcd54994a4b28e2336fd87f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "837731b5fcd54994a4b28e2336fd87f3" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42829 } } }
I20260808 13:08:39.141012 31100 sys_catalog.cc:458] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3 [sys.catalog]: This master's current role is: LEADER
I20260808 13:08:39.145336 31114 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:08:39.150279 31114 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:08:39.151993 31098 sys_catalog.cc:455] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "837731b5fcd54994a4b28e2336fd87f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "837731b5fcd54994a4b28e2336fd87f3" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 42829 } } }
I20260808 13:08:39.152699 31098 sys_catalog.cc:458] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3 [sys.catalog]: This master's current role is: LEADER
I20260808 13:08:39.162037 31114 catalog_manager.cc:1347] Generated new cluster ID: 66dce227c6b24c6ea250ef76fedd6122
I20260808 13:08:39.162194 31114 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:08:39.178846 31114 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:08:39.180611 31114 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:08:39.197530 31114 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3: Generated new TSK 0
I20260808 13:08:39.198853 31114 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:08:39.207623 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42829
--builtin_ntp_servers=127.1.124.148:42321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 13:08:39.605602 31124 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:39.605883 31124 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:39.606047 31124 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:39.613593 31124 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:39.613698 31124 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:39.613807 31124 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:08:39.623176 31124 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:42321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:39.624609 31124 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:39.626396 31124 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:39.641192 31134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:39.641211 31135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:39.645143 31137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:39.646184 31124 server_base.cc:1034] running on GCE node
I20260808 13:08:39.646956 31124 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:39.647922 31124 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:39.649145 31124 hybrid_clock.cc:648] HybridClock initialized: now 1786194519649091 us; error 42 us; skew 500 ppm
I20260808 13:08:39.649509 31124 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:08:39.651914 31124 webserver.cc:460] Webserver started at http://127.1.124.129:38563/ using document root <none> and password file <none>
I20260808 13:08:39.652693 31124 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:39.652818 31124 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:39.653195 31124 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:39.656255 31124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "ef84df5fd4f64d14a289a027af837cb8"
format_stamp: "Formatted at 2026-08-08 13:08:39 on dist-test-slave-t3q3"
server_key: "af5747c9ac724e47fd29e705fba8efbe"
server_key_iv: "fd2c71292b9e49893020d77a55577935"
server_key_version: "encryptionkey@0"
I20260808 13:08:39.657061 31124 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "ef84df5fd4f64d14a289a027af837cb8"
format_stamp: "Formatted at 2026-08-08 13:08:39 on dist-test-slave-t3q3"
server_key: "af5747c9ac724e47fd29e705fba8efbe"
server_key_iv: "fd2c71292b9e49893020d77a55577935"
server_key_version: "encryptionkey@0"
I20260808 13:08:39.662550 31124 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:08:39.666219 31148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:39.667589 31124 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:08:39.667769 31124 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "ef84df5fd4f64d14a289a027af837cb8"
format_stamp: "Formatted at 2026-08-08 13:08:39 on dist-test-slave-t3q3"
server_key: "af5747c9ac724e47fd29e705fba8efbe"
server_key_iv: "fd2c71292b9e49893020d77a55577935"
server_key_version: "encryptionkey@0"
I20260808 13:08:39.667996 31124 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:39.687382 31124 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:39.688424 31124 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:39.688771 31124 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:39.689853 31124 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:08:39.691438 31124 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:08:39.691530 31124 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:39.691660 31124 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:08:39.691761 31124 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:39.727819 31124 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:41597
I20260808 13:08:39.727878 31323 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:41597 every 8 connection(s)
I20260808 13:08:39.729362 31124 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:08:39.739125 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 31124
I20260808 13:08:39.739336 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:08:39.739795 1522 external_mini_cluster.cc:1428] Setting key 857d6de38658646dd703cd2fd182c594
I20260808 13:08:39.742786 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42829
--builtin_ntp_servers=127.1.124.148:42321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 13:08:39.743994 31326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:39.732424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:42997 (local address 127.1.124.190:42829)
0808 13:08:39.732912 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:39.732917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:39.734202 (+ 1285us) server_negotiation.cc:407] Connection header received
0808 13:08:39.736619 (+ 2417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:39.736623 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:39.736705 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:39.736809 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:39.737990 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:39.738827 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:39.740815 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:39.741397 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:39.741521 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:39.743102 (+ 1581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:39.743107 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:39.743110 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:39.743127 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:39.743178 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:39.743190 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:39.743297 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:39.743541 (+ 244us) server_negotiation.cc:299] Negotiation successful
0808 13:08:39.743642 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":263,"threads_started":1}
I20260808 13:08:39.745074 31327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:39.733091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:42829 (local address 127.1.124.129:42997)
0808 13:08:39.734003 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:39.734054 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:39.736323 (+ 2269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:39.737059 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:39.737067 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:39.737154 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:08:39.737814 (+ 660us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:39.737834 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:39.738997 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:39.739002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:39.740649 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:39.740666 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:39.741657 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:39.741667 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:39.741939 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:39.742701 (+ 762us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:08:39.742704 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:08:39.742745 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:08:39.742832 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:08:39.743303 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:08:39.743317 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:08:39.743326 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 13:08:39.743529 (+ 203us) client_negotiation.cc:240] Negotiation successful
0808 13:08:39.743952 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":268,"threads_started":1}
I20260808 13:08:39.746773 31324 heartbeater.cc:347] Connected to a master server at 127.1.124.190:42829
I20260808 13:08:39.747143 31324 heartbeater.cc:464] Registering TS with master...
I20260808 13:08:39.747918 31324 heartbeater.cc:511] Master 127.1.124.190:42829 requested a full tablet report, sending...
I20260808 13:08:39.750272 31039 ts_manager.cc:194] Registered new tserver with Master: ef84df5fd4f64d14a289a027af837cb8 (127.1.124.129:41597)
I20260808 13:08:39.752497 31039 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.124.129:42997
W20260808 13:08:40.097312 31328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:40.097721 31328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:40.097848 31328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:40.109808 31328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:40.109956 31328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:40.110117 31328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:08:40.120494 31328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:42321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:40.121817 31328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:40.124010 31328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:40.138121 31346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:40.138227 31340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:40.138247 31341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:40.138860 31328 server_base.cc:1034] running on GCE node
I20260808 13:08:40.139534 31328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:40.140694 31328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:40.141911 31328 hybrid_clock.cc:648] HybridClock initialized: now 1786194520141860 us; error 41 us; skew 500 ppm
I20260808 13:08:40.142256 31328 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:08:40.144627 31328 webserver.cc:460] Webserver started at http://127.1.124.130:37257/ using document root <none> and password file <none>
I20260808 13:08:40.145339 31328 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:40.145429 31328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:40.145812 31328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:40.148809 31328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "baa15083da2c4495a72d0b50e19050a2"
format_stamp: "Formatted at 2026-08-08 13:08:40 on dist-test-slave-t3q3"
server_key: "194396b996b40ede3ecd75ecfd6337b0"
server_key_iv: "d438d3739194e32d3324ad59b015c2cd"
server_key_version: "encryptionkey@0"
I20260808 13:08:40.149636 31328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "baa15083da2c4495a72d0b50e19050a2"
format_stamp: "Formatted at 2026-08-08 13:08:40 on dist-test-slave-t3q3"
server_key: "194396b996b40ede3ecd75ecfd6337b0"
server_key_iv: "d438d3739194e32d3324ad59b015c2cd"
server_key_version: "encryptionkey@0"
I20260808 13:08:40.155179 31328 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:08:40.159410 31353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:40.161027 31328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 13:08:40.161242 31328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "baa15083da2c4495a72d0b50e19050a2"
format_stamp: "Formatted at 2026-08-08 13:08:40 on dist-test-slave-t3q3"
server_key: "194396b996b40ede3ecd75ecfd6337b0"
server_key_iv: "d438d3739194e32d3324ad59b015c2cd"
server_key_version: "encryptionkey@0"
I20260808 13:08:40.161526 31328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:40.199090 31328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:40.200017 31328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:40.200407 31328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:40.201436 31328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:08:40.203111 31328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:08:40.203254 31328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:40.203384 31328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:08:40.203472 31328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:40.237725 31328 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:36201
I20260808 13:08:40.237833 31520 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:36201 every 8 connection(s)
I20260808 13:08:40.239152 31328 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:08:40.248138 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 31328
I20260808 13:08:40.248320 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:08:40.248793 1522 external_mini_cluster.cc:1428] Setting key 3369bc93bc9e24f414e75fc6d7491d9a
I20260808 13:08:40.252167 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42829
--builtin_ntp_servers=127.1.124.148:42321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 13:08:40.253041 31326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:40.241958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:55461 (local address 127.1.124.190:42829)
0808 13:08:40.242166 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:40.242170 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:40.243636 (+ 1466us) server_negotiation.cc:407] Connection header received
0808 13:08:40.245563 (+ 1927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:40.245569 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:40.245666 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:40.245783 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:40.246874 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.247608 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.249545 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.250167 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.250278 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:40.251979 (+ 1701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:40.251986 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:40.251991 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:40.252014 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:40.252055 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:40.252102 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:40.252209 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:40.252491 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 13:08:40.252606 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 13:08:40.253994 31523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:40.242517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:42829 (local address 127.1.124.130:55461)
0808 13:08:40.243459 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:40.243502 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:40.245267 (+ 1765us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:40.246031 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:40.246040 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:40.246126 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:08:40.246690 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:40.246707 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.247800 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.247804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:40.249382 (+ 1578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:40.249400 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.250413 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.250419 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:40.250758 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:40.251646 (+ 888us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:08:40.251648 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:08:40.251688 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:08:40.251771 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:08:40.252206 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:08:40.252215 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:08:40.252221 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 13:08:40.252463 (+ 242us) client_negotiation.cc:240] Negotiation successful
0808 13:08:40.252825 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":274,"threads_started":1}
I20260808 13:08:40.255486 31521 heartbeater.cc:347] Connected to a master server at 127.1.124.190:42829
I20260808 13:08:40.255825 31521 heartbeater.cc:464] Registering TS with master...
I20260808 13:08:40.256650 31521 heartbeater.cc:511] Master 127.1.124.190:42829 requested a full tablet report, sending...
I20260808 13:08:40.258152 31039 ts_manager.cc:194] Registered new tserver with Master: baa15083da2c4495a72d0b50e19050a2 (127.1.124.130:36201)
I20260808 13:08:40.258972 31039 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.124.130:55461
W20260808 13:08:40.606280 31525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:40.606577 31525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:40.606670 31525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:40.613902 31525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:40.614007 31525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:40.614113 31525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:08:40.623325 31525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:42321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:42829
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:40.624604 31525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:40.626447 31525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:40.639762 31536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:40.639936 31534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:40.640138 31539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:40.640920 31525 server_base.cc:1034] running on GCE node
I20260808 13:08:40.641544 31525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:40.642522 31525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:40.643756 31525 hybrid_clock.cc:648] HybridClock initialized: now 1786194520643696 us; error 59 us; skew 500 ppm
I20260808 13:08:40.644119 31525 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:08:40.646803 31525 webserver.cc:460] Webserver started at http://127.1.124.131:32785/ using document root <none> and password file <none>
I20260808 13:08:40.647535 31525 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:40.647679 31525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:40.648196 31525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:40.651173 31525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "0c487b3091204d6db68d42e3f2d72e84"
format_stamp: "Formatted at 2026-08-08 13:08:40 on dist-test-slave-t3q3"
server_key: "d3df00f027acce7e1f4c3bb56a0fb866"
server_key_iv: "191772eab6b51ab7a68c1b93f6c33f54"
server_key_version: "encryptionkey@0"
I20260808 13:08:40.651989 31525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "0c487b3091204d6db68d42e3f2d72e84"
format_stamp: "Formatted at 2026-08-08 13:08:40 on dist-test-slave-t3q3"
server_key: "d3df00f027acce7e1f4c3bb56a0fb866"
server_key_iv: "191772eab6b51ab7a68c1b93f6c33f54"
server_key_version: "encryptionkey@0"
I20260808 13:08:40.657320 31525 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260808 13:08:40.660938 31549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:40.662343 31525 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:08:40.662535 31525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "0c487b3091204d6db68d42e3f2d72e84"
format_stamp: "Formatted at 2026-08-08 13:08:40 on dist-test-slave-t3q3"
server_key: "d3df00f027acce7e1f4c3bb56a0fb866"
server_key_iv: "191772eab6b51ab7a68c1b93f6c33f54"
server_key_version: "encryptionkey@0"
I20260808 13:08:40.662770 31525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:40.676446 31525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:40.677312 31525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:40.677642 31525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:40.678668 31525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:08:40.680300 31525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:08:40.680423 31525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:40.680572 31525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:08:40.680665 31525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:40.715710 31525 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:45199
I20260808 13:08:40.715767 31713 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:45199 every 8 connection(s)
I20260808 13:08:40.717105 31525 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:08:40.717542 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 31525
I20260808 13:08:40.717739 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:08:40.718398 1522 external_mini_cluster.cc:1428] Setting key f9f52ada0d86e4543566119f4025924c
I20260808 13:08:40.736228 31326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:40.719912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:53569 (local address 127.1.124.190:42829)
0808 13:08:40.720178 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:40.720187 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:40.721892 (+ 1705us) server_negotiation.cc:407] Connection header received
0808 13:08:40.723704 (+ 1812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:40.723708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:40.723774 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:40.723877 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:40.725134 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.725869 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.727796 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.728355 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.728493 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:40.735098 (+ 6605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:40.735105 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:40.735110 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:40.735136 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:40.735182 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:40.735202 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:40.735314 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:40.735661 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 13:08:40.735865 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 13:08:40.737730 31717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:40.720632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:42829 (local address 127.1.124.131:53569)
0808 13:08:40.721703 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:40.721756 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:40.723434 (+ 1678us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:40.724221 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:40.724229 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:40.724329 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:08:40.724940 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:40.724959 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.726052 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.726056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:40.727643 (+ 1587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:40.727660 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.728633 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.728642 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:40.728973 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:40.734572 (+ 5599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:08:40.734576 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:08:40.734645 (+ 69us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:08:40.734769 (+ 124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:08:40.735320 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:08:40.735333 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:08:40.735365 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 13:08:40.735620 (+ 255us) client_negotiation.cc:240] Negotiation successful
0808 13:08:40.736152 (+ 532us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"thread_start_us":366,"threads_started":1}
I20260808 13:08:40.739328 31715 heartbeater.cc:347] Connected to a master server at 127.1.124.190:42829
I20260808 13:08:40.739696 31715 heartbeater.cc:464] Registering TS with master...
I20260808 13:08:40.740619 31715 heartbeater.cc:511] Master 127.1.124.190:42829 requested a full tablet report, sending...
I20260808 13:08:40.742110 31039 ts_manager.cc:194] Registered new tserver with Master: 0c487b3091204d6db68d42e3f2d72e84 (127.1.124.131:45199)
I20260808 13:08:40.742889 31039 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.124.131:53569
I20260808 13:08:40.747412 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:08:40.755667 31324 heartbeater.cc:503] Master 127.1.124.190:42829 was elected leader, sending a full tablet report...
I20260808 13:08:40.758545 31326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:40.752117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34698 (local address 127.1.124.190:42829)
0808 13:08:40.752411 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:40.752417 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:40.752674 (+ 257us) server_negotiation.cc:407] Connection header received
0808 13:08:40.752877 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:40.752881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:40.752946 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:40.753066 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:40.753612 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.754359 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.755812 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.756335 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.756500 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:40.757705 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:40.757710 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:40.757713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:40.757729 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:40.757761 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:40.757772 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:40.757889 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:40.758189 (+ 300us) server_negotiation.cc:299] Negotiation successful
0808 13:08:40.758260 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260808 13:08:40.762298 31039 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34698:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:08:40.769513 31039 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:08:40.795382 31730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:40.782565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:41597 (local address 127.0.0.1:42972)
0808 13:08:40.783449 (+ 884us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:40.783504 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:40.783701 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:40.785160 (+ 1459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:40.785172 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:40.785215 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:40.785765 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:40.785782 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.788680 (+ 2898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.788689 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:40.791861 (+ 3172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:40.791881 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.794281 (+ 2400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.794288 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:40.794570 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:40.794658 (+ 88us) client_negotiation.cc:769] Sending connection context
0808 13:08:40.794785 (+ 127us) client_negotiation.cc:240] Negotiation successful
0808 13:08:40.794988 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"thread_start_us":372,"threads_started":1}
I20260808 13:08:40.796226 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:40.782633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42972 (local address 127.1.124.129:41597)
0808 13:08:40.784345 (+ 1712us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:40.784357 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:40.784396 (+ 39us) server_negotiation.cc:407] Connection header received
0808 13:08:40.784542 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:40.784553 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:40.784848 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:40.786029 (+ 1181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:40.786372 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.788474 (+ 2102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.792212 (+ 3738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.794042 (+ 1830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.795109 (+ 1067us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:40.795469 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:40.795579 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 13:08:40.795776 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1538,"thread_start_us":1429,"threads_started":1}
I20260808 13:08:40.800577 31219 tablet_service.cc:1467] Processing CreateTablet for tablet 0e8374765c63424ba3b7ae5f5d2b2d64 (DEFAULT_TABLE table=test-table [id=ece18af693e7422b83a70258677e1670]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:08:40.802270 31733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:40.783237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:36201 (local address 127.0.0.1:46702)
0808 13:08:40.793771 (+ 10534us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:40.793794 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:40.793935 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:40.794682 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:40.794687 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:40.794719 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:40.795492 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:40.795510 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.797468 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.797474 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:40.799141 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:40.799156 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.801353 (+ 2197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.801360 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:40.801588 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:40.801627 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 13:08:40.801745 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 13:08:40.801908 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10411,"thread_start_us":9434,"threads_started":1}
I20260808 13:08:40.802347 31735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:40.784628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46702 (local address 127.1.124.130:36201)
0808 13:08:40.791937 (+ 7309us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:40.791946 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:40.794074 (+ 2128us) server_negotiation.cc:407] Connection header received
0808 13:08:40.794194 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:40.794205 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:40.794446 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:40.794676 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:40.795722 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.797274 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.799369 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.800806 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.800954 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:40.801205 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:40.801816 (+ 611us) server_negotiation.cc:299] Negotiation successful
0808 13:08:40.802010 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7124,"thread_start_us":6983,"threads_started":1}
I20260808 13:08:40.802321 31219 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e8374765c63424ba3b7ae5f5d2b2d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:40.803464 31734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:40.783159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:45199 (local address 127.0.0.1:35052)
0808 13:08:40.793032 (+ 9873us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:40.793051 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:40.793228 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:40.794231 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:40.794239 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:40.794272 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:40.794714 (+ 442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:40.794726 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.797742 (+ 3016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.797760 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:40.799904 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:40.799920 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.802343 (+ 2423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.802352 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:40.802760 (+ 408us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:40.802799 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 13:08:40.802920 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 13:08:40.803072 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9686,"mutex_wait_us":1,"thread_start_us":10335,"threads_started":1}
I20260808 13:08:40.803537 31737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:40.783891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35052 (local address 127.1.124.131:45199)
0808 13:08:40.792614 (+ 8723us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:40.792622 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:40.793175 (+ 553us) server_negotiation.cc:407] Connection header received
0808 13:08:40.793425 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:40.793437 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:40.793756 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:40.793984 (+ 228us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:40.794946 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.797553 (+ 2607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.800169 (+ 2616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.801749 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.801884 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:40.802184 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:40.802971 (+ 787us) server_negotiation.cc:299] Negotiation successful
0808 13:08:40.803151 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8528,"thread_start_us":8345,"threads_started":1}
I20260808 13:08:40.806206 31424 tablet_service.cc:1467] Processing CreateTablet for tablet 0e8374765c63424ba3b7ae5f5d2b2d64 (DEFAULT_TABLE table=test-table [id=ece18af693e7422b83a70258677e1670]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:08:40.807680 31424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e8374765c63424ba3b7ae5f5d2b2d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:40.807256 31618 tablet_service.cc:1467] Processing CreateTablet for tablet 0e8374765c63424ba3b7ae5f5d2b2d64 (DEFAULT_TABLE table=test-table [id=ece18af693e7422b83a70258677e1670]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:08:40.808636 31618 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e8374765c63424ba3b7ae5f5d2b2d64. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:40.818905 31738 tablet_bootstrap.cc:492] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8: Bootstrap starting.
I20260808 13:08:40.822850 31738 tablet_bootstrap.cc:654] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:40.823460 31742 tablet_bootstrap.cc:492] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2: Bootstrap starting.
I20260808 13:08:40.824424 31743 tablet_bootstrap.cc:492] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84: Bootstrap starting.
I20260808 13:08:40.825053 31738 log.cc:824] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:40.828819 31743 tablet_bootstrap.cc:654] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:40.829541 31742 tablet_bootstrap.cc:654] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:40.830317 31743 log.cc:824] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:40.830618 31738 tablet_bootstrap.cc:492] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8: No bootstrap required, opened a new log
I20260808 13:08:40.830916 31738 ts_tablet_manager.cc:1397] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260808 13:08:40.830909 31742 log.cc:824] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:40.834035 31743 tablet_bootstrap.cc:492] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84: No bootstrap required, opened a new log
I20260808 13:08:40.834218 31742 tablet_bootstrap.cc:492] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2: No bootstrap required, opened a new log
I20260808 13:08:40.834333 31743 ts_tablet_manager.cc:1397] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.006s
I20260808 13:08:40.834586 31742 ts_tablet_manager.cc:1397] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260808 13:08:40.841943 31738 raft_consensus.cc:348] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef84df5fd4f64d14a289a027af837cb8" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41597 } } peers { permanent_uuid: "0c487b3091204d6db68d42e3f2d72e84" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 45199 } } peers { permanent_uuid: "baa15083da2c4495a72d0b50e19050a2" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36201 } }
I20260808 13:08:40.842449 31738 raft_consensus.cc:374] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:40.842561 31738 raft_consensus.cc:729] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef84df5fd4f64d14a289a027af837cb8, State: Initialized, Role: FOLLOWER
I20260808 13:08:40.843304 31738 consensus_queue.cc:260] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef84df5fd4f64d14a289a027af837cb8" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41597 } } peers { permanent_uuid: "0c487b3091204d6db68d42e3f2d72e84" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 45199 } } peers { permanent_uuid: "baa15083da2c4495a72d0b50e19050a2" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36201 } }
I20260808 13:08:40.843897 31743 raft_consensus.cc:348] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef84df5fd4f64d14a289a027af837cb8" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41597 } } peers { permanent_uuid: "0c487b3091204d6db68d42e3f2d72e84" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 45199 } } peers { permanent_uuid: "baa15083da2c4495a72d0b50e19050a2" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36201 } }
I20260808 13:08:40.845252 31738 ts_tablet_manager.cc:1428] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260808 13:08:40.844431 31743 raft_consensus.cc:374] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:40.845332 31742 raft_consensus.cc:348] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef84df5fd4f64d14a289a027af837cb8" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41597 } } peers { permanent_uuid: "0c487b3091204d6db68d42e3f2d72e84" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 45199 } } peers { permanent_uuid: "baa15083da2c4495a72d0b50e19050a2" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36201 } }
I20260808 13:08:40.845683 31743 raft_consensus.cc:729] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0c487b3091204d6db68d42e3f2d72e84, State: Initialized, Role: FOLLOWER
I20260808 13:08:40.845747 31742 raft_consensus.cc:374] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:40.845834 31742 raft_consensus.cc:729] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa15083da2c4495a72d0b50e19050a2, State: Initialized, Role: FOLLOWER
I20260808 13:08:40.846391 31743 consensus_queue.cc:260] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef84df5fd4f64d14a289a027af837cb8" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41597 } } peers { permanent_uuid: "0c487b3091204d6db68d42e3f2d72e84" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 45199 } } peers { permanent_uuid: "baa15083da2c4495a72d0b50e19050a2" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36201 } }
I20260808 13:08:40.846392 31742 consensus_queue.cc:260] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef84df5fd4f64d14a289a027af837cb8" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41597 } } peers { permanent_uuid: "0c487b3091204d6db68d42e3f2d72e84" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 45199 } } peers { permanent_uuid: "baa15083da2c4495a72d0b50e19050a2" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36201 } }
I20260808 13:08:40.848690 31521 heartbeater.cc:503] Master 127.1.124.190:42829 was elected leader, sending a full tablet report...
I20260808 13:08:40.848925 31743 ts_tablet_manager.cc:1428] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84: Time spent starting tablet: real 0.014s user 0.011s sys 0.001s
I20260808 13:08:40.849300 31742 ts_tablet_manager.cc:1428] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2: Time spent starting tablet: real 0.015s user 0.009s sys 0.005s
I20260808 13:08:40.849514 31715 heartbeater.cc:503] Master 127.1.124.190:42829 was elected leader, sending a full tablet report...
I20260808 13:08:40.947031 31747 raft_consensus.cc:482] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:08:40.947377 31747 raft_consensus.cc:504] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef84df5fd4f64d14a289a027af837cb8" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41597 } } peers { permanent_uuid: "0c487b3091204d6db68d42e3f2d72e84" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 45199 } } peers { permanent_uuid: "baa15083da2c4495a72d0b50e19050a2" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36201 } }
I20260808 13:08:40.949198 31747 leader_election.cc:288] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0c487b3091204d6db68d42e3f2d72e84 (127.1.124.131:45199), baa15083da2c4495a72d0b50e19050a2 (127.1.124.130:36201)
I20260808 13:08:40.959355 31737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:40.949826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:34203 (local address 127.1.124.131:45199)
0808 13:08:40.950027 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:40.950035 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:40.950937 (+ 902us) server_negotiation.cc:407] Connection header received
0808 13:08:40.951207 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:40.951212 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:40.951290 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:40.951378 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:40.952512 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.953671 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.956944 (+ 3273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.958340 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.958503 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:40.958636 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:40.958971 (+ 335us) server_negotiation.cc:299] Negotiation successful
0808 13:08:40.959127 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 13:08:40.959357 31735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:40.949705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:39145 (local address 127.1.124.130:36201)
0808 13:08:40.949922 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:40.949930 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:40.950853 (+ 923us) server_negotiation.cc:407] Connection header received
0808 13:08:40.951146 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:40.951154 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:40.951251 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:40.951358 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:40.952513 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.953671 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.956944 (+ 3273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.958166 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.958342 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:40.958428 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:40.958939 (+ 511us) server_negotiation.cc:299] Negotiation successful
0808 13:08:40.959078 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 13:08:40.959760 31753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:40.949734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:45199 (local address 127.1.124.129:34203)
0808 13:08:40.950834 (+ 1100us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:40.950854 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:40.950995 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:40.951544 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:40.951549 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:40.951593 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:40.952294 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:40.952310 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.953824 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.953829 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:40.956696 (+ 2867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:40.956712 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.958627 (+ 1915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.958633 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:40.958790 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:40.958830 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 13:08:40.958910 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 13:08:40.959006 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":935,"mutex_wait_us":179,"thread_start_us":489,"threads_started":1}
I20260808 13:08:40.959782 31752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:40.949628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:36201 (local address 127.1.124.129:39145)
0808 13:08:40.950719 (+ 1091us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:40.950735 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:40.950926 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:40.951393 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:40.951398 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:40.951433 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:40.952298 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:40.952310 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.953823 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.953829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:40.956696 (+ 2867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:40.956712 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.958376 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.958380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:40.958695 (+ 315us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:40.958732 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 13:08:40.958838 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 13:08:40.958989 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":954,"mutex_wait_us":258,"spinlock_wait_cycles":23808,"thread_start_us":472,"threads_started":1}
I20260808 13:08:40.960870 31750 raft_consensus.cc:482] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:08:40.960988 31457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e8374765c63424ba3b7ae5f5d2b2d64" candidate_uuid: "ef84df5fd4f64d14a289a027af837cb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa15083da2c4495a72d0b50e19050a2" is_pre_election: true
I20260808 13:08:40.961033 31645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e8374765c63424ba3b7ae5f5d2b2d64" candidate_uuid: "ef84df5fd4f64d14a289a027af837cb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c487b3091204d6db68d42e3f2d72e84" is_pre_election: true
I20260808 13:08:40.961213 31750 raft_consensus.cc:504] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef84df5fd4f64d14a289a027af837cb8" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41597 } } peers { permanent_uuid: "0c487b3091204d6db68d42e3f2d72e84" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 45199 } } peers { permanent_uuid: "baa15083da2c4495a72d0b50e19050a2" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36201 } }
I20260808 13:08:40.961623 31457 raft_consensus.cc:2454] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef84df5fd4f64d14a289a027af837cb8 in term 0.
I20260808 13:08:40.962024 31645 raft_consensus.cc:2454] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef84df5fd4f64d14a289a027af837cb8 in term 0.
I20260808 13:08:40.962335 31155 leader_election.cc:302] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: baa15083da2c4495a72d0b50e19050a2, ef84df5fd4f64d14a289a027af837cb8; no voters:
I20260808 13:08:40.962786 31747 raft_consensus.cc:2781] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:08:40.962971 31747 raft_consensus.cc:482] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:08:40.963102 31747 raft_consensus.cc:3037] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:40.963304 31750 leader_election.cc:288] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ef84df5fd4f64d14a289a027af837cb8 (127.1.124.129:41597), baa15083da2c4495a72d0b50e19050a2 (127.1.124.130:36201)
I20260808 13:08:40.966540 31747 raft_consensus.cc:504] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef84df5fd4f64d14a289a027af837cb8" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41597 } } peers { permanent_uuid: "0c487b3091204d6db68d42e3f2d72e84" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 45199 } } peers { permanent_uuid: "baa15083da2c4495a72d0b50e19050a2" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36201 } }
I20260808 13:08:40.967322 31747 leader_election.cc:288] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8 [CANDIDATE]: Term 1 election: Requested vote from peers 0c487b3091204d6db68d42e3f2d72e84 (127.1.124.131:45199), baa15083da2c4495a72d0b50e19050a2 (127.1.124.130:36201)
I20260808 13:08:40.967760 31645 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e8374765c63424ba3b7ae5f5d2b2d64" candidate_uuid: "ef84df5fd4f64d14a289a027af837cb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0c487b3091204d6db68d42e3f2d72e84"
I20260808 13:08:40.967743 31457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e8374765c63424ba3b7ae5f5d2b2d64" candidate_uuid: "ef84df5fd4f64d14a289a027af837cb8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa15083da2c4495a72d0b50e19050a2"
I20260808 13:08:40.967960 31645 raft_consensus.cc:3037] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:40.967979 31457 raft_consensus.cc:3037] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2 [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:08:40.968976 31716 tablet.cc:2369] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:08:40.970880 31717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:40.963444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:41597 (local address 127.1.124.131:32943)
0808 13:08:40.963633 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:40.963654 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:40.963778 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:40.964485 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:40.964490 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:40.964516 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:40.964715 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:40.964726 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.966156 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.966161 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:40.968291 (+ 2130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:40.968308 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.970065 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.970071 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:40.970303 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:40.970342 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 13:08:40.970451 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 13:08:40.970597 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 13:08:40.971498 31457 raft_consensus.cc:2454] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef84df5fd4f64d14a289a027af837cb8 in term 1.
I20260808 13:08:40.971668 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:40.963846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:32943 (local address 127.1.124.129:41597)
0808 13:08:40.964023 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:40.964028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:40.964045 (+ 17us) server_negotiation.cc:407] Connection header received
0808 13:08:40.964128 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:40.964132 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:40.964215 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:40.964312 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:40.964897 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.965983 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.968482 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.969683 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.969846 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:40.969917 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:40.971246 (+ 1329us) server_negotiation.cc:299] Negotiation successful
0808 13:08:40.971431 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 13:08:40.971251 31645 raft_consensus.cc:2454] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef84df5fd4f64d14a289a027af837cb8 in term 1.
I20260808 13:08:40.972184 31155 leader_election.cc:302] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: baa15083da2c4495a72d0b50e19050a2, ef84df5fd4f64d14a289a027af837cb8; no voters:
I20260808 13:08:40.972493 31747 raft_consensus.cc:2781] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:08:40.973038 31747 raft_consensus.cc:686] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8 [term 1 LEADER]: Becoming Leader. State: Replica: ef84df5fd4f64d14a289a027af837cb8, State: Running, Role: LEADER
I20260808 13:08:40.972807 31257 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e8374765c63424ba3b7ae5f5d2b2d64" candidate_uuid: "0c487b3091204d6db68d42e3f2d72e84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef84df5fd4f64d14a289a027af837cb8" is_pre_election: true
I20260808 13:08:40.973604 31747 consensus_queue.cc:237] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef84df5fd4f64d14a289a027af837cb8" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41597 } } peers { permanent_uuid: "0c487b3091204d6db68d42e3f2d72e84" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 45199 } } peers { permanent_uuid: "baa15083da2c4495a72d0b50e19050a2" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36201 } }
I20260808 13:08:40.977403 31735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:40.964194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:35519 (local address 127.1.124.130:36201)
0808 13:08:40.964362 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:40.964367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:40.964464 (+ 97us) server_negotiation.cc:407] Connection header received
0808 13:08:40.964739 (+ 275us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:40.964744 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:40.964824 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:40.964925 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:40.965522 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.966374 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.975281 (+ 8907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.976385 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.976566 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:40.976648 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:40.977002 (+ 354us) server_negotiation.cc:299] Negotiation successful
0808 13:08:40.977146 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 13:08:40.977470 31754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:40.963688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:36201 (local address 127.1.124.131:35519)
0808 13:08:40.964375 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:40.964392 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:40.964617 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:40.964942 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:40.964946 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:40.964970 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:40.965338 (+ 368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:40.965351 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.966553 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.966563 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:40.975080 (+ 8517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:40.975097 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.976575 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.976581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:40.976790 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:40.976837 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 13:08:40.976957 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 13:08:40.977136 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"spinlock_wait_cycles":11520,"thread_start_us":738,"threads_started":1}
I20260808 13:08:40.978173 31457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e8374765c63424ba3b7ae5f5d2b2d64" candidate_uuid: "0c487b3091204d6db68d42e3f2d72e84" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa15083da2c4495a72d0b50e19050a2" is_pre_election: true
I20260808 13:08:40.978477 31457 raft_consensus.cc:2379] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0c487b3091204d6db68d42e3f2d72e84 in current term 1: Already voted for candidate ef84df5fd4f64d14a289a027af837cb8 in this term.
I20260808 13:08:40.979135 31554 leader_election.cc:302] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0c487b3091204d6db68d42e3f2d72e84; no voters: baa15083da2c4495a72d0b50e19050a2, ef84df5fd4f64d14a289a027af837cb8
I20260808 13:08:40.979538 31750 raft_consensus.cc:2726] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260808 13:08:40.979104 31037 catalog_manager.cc:5495] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef84df5fd4f64d14a289a027af837cb8 (127.1.124.129). New cstate: current_term: 1 leader_uuid: "ef84df5fd4f64d14a289a027af837cb8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ef84df5fd4f64d14a289a027af837cb8" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0c487b3091204d6db68d42e3f2d72e84" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 45199 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "baa15083da2c4495a72d0b50e19050a2" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36201 } health_report { overall_health: UNKNOWN } } }
W20260808 13:08:40.982235 31325 tablet.cc:2369] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 13:08:40.991348 31522 tablet.cc:2369] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:08:41.001468 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:40.996019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42976 (local address 127.1.124.129:41597)
0808 13:08:40.996246 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:40.996250 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:40.996270 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:08:40.996375 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:40.996379 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:40.996488 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:40.996615 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:08:40.997019 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.997762 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.999187 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:40.999607 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:40.999709 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:41.000310 (+ 601us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:08:41.000813 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:08:41.000947 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:41.001217 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 13:08:41.001254 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260808 13:08:41.009526 31457 raft_consensus.cc:1261] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2 [term 1 FOLLOWER]: Refusing update from remote peer ef84df5fd4f64d14a289a027af837cb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:08:41.009665 31645 raft_consensus.cc:1261] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84 [term 1 FOLLOWER]: Refusing update from remote peer ef84df5fd4f64d14a289a027af837cb8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:08:41.010497 31757 consensus_queue.cc:1035] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "baa15083da2c4495a72d0b50e19050a2" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 36201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:08:41.010839 31747 consensus_queue.cc:1035] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0c487b3091204d6db68d42e3f2d72e84" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 45199 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:08:41.021993 31762 mvcc.cc:205] Tried to move back new op lower bound from 7316252758045552640 to 7316252757912494080. Current Snapshot: MvccSnapshot[applied={T|T < 7316252758045552640}]
I20260808 13:08:41.036445 31037 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:08:41.036746 31037 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34698:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:08:41.040947 31037 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 837731b5fcd54994a4b28e2336fd87f3: Sending DeleteTablet for 3 replicas of tablet 0e8374765c63424ba3b7ae5f5d2b2d64
I20260808 13:08:41.042250 31219 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e8374765c63424ba3b7ae5f5d2b2d64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:08:41 UTC) from {username='slave'} at 127.0.0.1:42972
I20260808 13:08:41.042243 31618 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e8374765c63424ba3b7ae5f5d2b2d64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:08:41 UTC) from {username='slave'} at 127.0.0.1:35052
I20260808 13:08:41.043084 31424 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e8374765c63424ba3b7ae5f5d2b2d64 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:08:41 UTC) from {username='slave'} at 127.0.0.1:46702
I20260808 13:08:41.043243 31778 tablet_replica.cc:330] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84: stopping tablet replica
I20260808 13:08:41.043800 31778 raft_consensus.cc:2229] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:08:41.044412 31778 raft_consensus.cc:2258] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:08:41.045372 31776 tablet_replica.cc:330] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8: stopping tablet replica
I20260808 13:08:41.045779 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 31124
I20260808 13:08:41.045853 31776 raft_consensus.cc:2229] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:08:41.046684 31776 raft_consensus.cc:2258] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:08:41.048854 31778 ts_tablet_manager.cc:1905] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:08:41.053943 31776 ts_tablet_manager.cc:1905] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:08:41.062500 31776 ts_tablet_manager.cc:1918] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:08:41.062796 31776 log.cc:1196] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/0e8374765c63424ba3b7ae5f5d2b2d64
I20260808 13:08:41.062983 31779 tablet_replica.cc:330] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2: stopping tablet replica
I20260808 13:08:41.063472 31779 raft_consensus.cc:2229] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:08:41.063722 31776 ts_tablet_manager.cc:1939] T 0e8374765c63424ba3b7ae5f5d2b2d64 P ef84df5fd4f64d14a289a027af837cb8: Deleting consensus metadata
I20260808 13:08:41.064090 31779 raft_consensus.cc:2258] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:08:41.065800 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:41.047654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42982 (local address 127.1.124.129:41597)
0808 13:08:41.047845 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:41.047851 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:41.056613 (+ 8762us) server_negotiation.cc:407] Connection header received
0808 13:08:41.056816 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:41.056824 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:41.056950 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:41.057058 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:41.057681 (+ 623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:41.058783 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:41.061026 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:41.062107 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:41.062236 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:41.063894 (+ 1658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:41.063907 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:41.063925 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:41.064835 (+ 910us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:41.064899 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:41.064946 (+ 47us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:41.065026 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:41.065313 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 13:08:41.065421 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 13:08:41.065927 31778 ts_tablet_manager.cc:1918] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:08:41.065951 31014 catalog_manager.cc:4841] TS ef84df5fd4f64d14a289a027af837cb8 (127.1.124.129:41597): tablet 0e8374765c63424ba3b7ae5f5d2b2d64 (table test-table [id=ece18af693e7422b83a70258677e1670]) successfully deleted
I20260808 13:08:41.066883 31778 log.cc:1196] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/0e8374765c63424ba3b7ae5f5d2b2d64
I20260808 13:08:41.067221 31317 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:41.067723 31778 ts_tablet_manager.cc:1939] T 0e8374765c63424ba3b7ae5f5d2b2d64 P 0c487b3091204d6db68d42e3f2d72e84: Deleting consensus metadata
I20260808 13:08:41.069484 31779 ts_tablet_manager.cc:1905] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:08:41.070123 31016 catalog_manager.cc:4841] TS 0c487b3091204d6db68d42e3f2d72e84 (127.1.124.131:45199): tablet 0e8374765c63424ba3b7ae5f5d2b2d64 (table test-table [id=ece18af693e7422b83a70258677e1670]) successfully deleted
I20260808 13:08:41.076869 31779 ts_tablet_manager.cc:1918] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:08:41.077061 31779 log.cc:1196] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/0e8374765c63424ba3b7ae5f5d2b2d64
I20260808 13:08:41.077620 31779 ts_tablet_manager.cc:1939] T 0e8374765c63424ba3b7ae5f5d2b2d64 P baa15083da2c4495a72d0b50e19050a2: Deleting consensus metadata
I20260808 13:08:41.079473 31018 catalog_manager.cc:4841] TS baa15083da2c4495a72d0b50e19050a2 (127.1.124.130:36201): tablet 0e8374765c63424ba3b7ae5f5d2b2d64 (table test-table [id=ece18af693e7422b83a70258677e1670]) successfully deleted
I20260808 13:08:41.182212 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 31124
I20260808 13:08:41.208233 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 31328
I20260808 13:08:41.214783 31735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:41.209117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46708 (local address 127.1.124.130:36201)
0808 13:08:41.209272 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:41.209277 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:41.209295 (+ 18us) server_negotiation.cc:407] Connection header received
0808 13:08:41.209377 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:41.209381 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:41.209454 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:41.209537 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:41.209851 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:41.210565 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:41.211803 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:41.212331 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:41.212429 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:41.213465 (+ 1036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:41.213473 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:41.213486 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:41.214071 (+ 585us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:41.214116 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:41.214136 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:41.214224 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:41.214368 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 13:08:41.214451 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 13:08:41.215608 31516 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:41.304996 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 31328
I20260808 13:08:41.331136 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 31525
I20260808 13:08:41.339425 31737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:41.332018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35058 (local address 127.1.124.131:45199)
0808 13:08:41.332268 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:41.332273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:41.332310 (+ 37us) server_negotiation.cc:407] Connection header received
0808 13:08:41.332493 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:41.332497 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:41.332588 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:41.332675 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:41.333160 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:41.333879 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:41.335342 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:41.335818 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:41.335994 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:41.337608 (+ 1614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:41.337621 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:41.337642 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:41.338519 (+ 877us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:41.338580 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:41.338603 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:41.338739 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:41.338922 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 13:08:41.339054 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136}
I20260808 13:08:41.340233 31708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:41.430560 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 31525
I20260808 13:08:41.455700 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 30994
I20260808 13:08:41.457003 31085 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:41.567080 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 30994
2026-08-08T13:08:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3043 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:08:41 dist-test-slave-t3q3 krb5kdc[31791](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:08:41 dist-test-slave-t3q3 krb5kdc[31791](info): set up 2 sockets
Aug 08 13:08:41 dist-test-slave-t3q3 krb5kdc[31791](info): commencing operation
krb5kdc: starting...
W20260808 13:08:42.920307 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.010s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:08:43 dist-test-slave-t3q3 krb5kdc[31791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:08:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:08:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:08:43.250028 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:41517
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:44293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:41517
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:08:43.608120 31811 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:43.608421 31811 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:43.608608 31811 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:43.615886 31811 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:08:43.616004 31811 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:43.616101 31811 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:43.616194 31811 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:08:43.616278 31811 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:08:43.626585 31811 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:44293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:41517
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:41517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:43.628001 31811 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:43.629360 31811 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:43.641570 31825 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:43.641717 31818 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:43.641748 31819 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:43.642360 31811 server_base.cc:1034] running on GCE node
I20260808 13:08:43.643234 31811 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:43.644774 31811 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:43.646029 31811 hybrid_clock.cc:648] HybridClock initialized: now 1786194523645977 us; error 38 us; skew 500 ppm
I20260808 13:08:43.646389 31811 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:08:43 dist-test-slave-t3q3 krb5kdc[31791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194523, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:08:43.649621 31811 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:08:43.651291 31811 webserver.cc:460] Webserver started at http://127.1.124.190:36507/ using document root <none> and password file <none>
I20260808 13:08:43.652009 31811 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:43.652209 31811 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:43.652546 31811 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:43.655774 31811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "cd5fdb779ad846ad8afeb064ab59999e"
format_stamp: "Formatted at 2026-08-08 13:08:43 on dist-test-slave-t3q3"
server_key: "7a6f6d534d96ac1da5aeca378b6f38b5"
server_key_iv: "0f38e874d481a67147227bc722e6e29a"
server_key_version: "encryptionkey@0"
I20260808 13:08:43.656661 31811 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "cd5fdb779ad846ad8afeb064ab59999e"
format_stamp: "Formatted at 2026-08-08 13:08:43 on dist-test-slave-t3q3"
server_key: "7a6f6d534d96ac1da5aeca378b6f38b5"
server_key_iv: "0f38e874d481a67147227bc722e6e29a"
server_key_version: "encryptionkey@0"
I20260808 13:08:43.662098 31811 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:08:43.665782 31835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:43.667176 31811 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:08:43.667389 31811 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "cd5fdb779ad846ad8afeb064ab59999e"
format_stamp: "Formatted at 2026-08-08 13:08:43 on dist-test-slave-t3q3"
server_key: "7a6f6d534d96ac1da5aeca378b6f38b5"
server_key_iv: "0f38e874d481a67147227bc722e6e29a"
server_key_version: "encryptionkey@0"
I20260808 13:08:43.667604 31811 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:43.687619 31811 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:43.692693 31811 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:43.693084 31811 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:43.715126 31811 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:41517
I20260808 13:08:43.715231 31905 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:41517 every 8 connection(s)
I20260808 13:08:43.716732 31811 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:08:43.721095 31906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:43.725744 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 31811
I20260808 13:08:43.725991 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:08:43.726498 1522 external_mini_cluster.cc:1428] Setting key 5045477967bc86378f84e01da145129f
I20260808 13:08:43.734320 31906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e: Bootstrap starting.
I20260808 13:08:43.738654 31906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:43.739967 31906 log.cc:824] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:43.743949 31906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e: No bootstrap required, opened a new log
Aug 08 13:08:43 dist-test-slave-t3q3 krb5kdc[31791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:08:43.754871 31914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:43.728853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48778 (local address 127.1.124.190:41517)
0808 13:08:43.729747 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:43.729760 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:43.729807 (+ 47us) server_negotiation.cc:407] Connection header received
0808 13:08:43.732339 (+ 2532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:43.732367 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:43.732797 (+ 430us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:43.733183 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:43.735431 (+ 2248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:43.737065 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:43.739183 (+ 2118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:43.739966 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:43.740262 (+ 296us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:43.745495 (+ 5233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:43.745508 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:43.745534 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:43.745594 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:43.749863 (+ 4269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:43.750654 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:43.750664 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:43.750675 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:43.750908 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:43.751347 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:43.751353 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:43.751356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:43.752165 (+ 809us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:43.752445 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:43.752941 (+ 496us) server_negotiation.cc:299] Negotiation successful
0808 13:08:43.753443 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":419,"threads_started":1}
I20260808 13:08:43.755263 31906 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd5fdb779ad846ad8afeb064ab59999e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41517 } }
I20260808 13:08:43.755664 31906 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:43.755777 31906 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd5fdb779ad846ad8afeb064ab59999e, State: Initialized, Role: FOLLOWER
I20260808 13:08:43.756458 31906 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd5fdb779ad846ad8afeb064ab59999e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41517 } }
I20260808 13:08:43.756734 31906 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:08:43.756896 31906 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:08:43.757105 31906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:43.759443 31906 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd5fdb779ad846ad8afeb064ab59999e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41517 } }
I20260808 13:08:43.759950 31906 leader_election.cc:302] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd5fdb779ad846ad8afeb064ab59999e; no voters:
I20260808 13:08:43.760576 31906 leader_election.cc:288] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:08:43.760735 31918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:08:43.761443 31918 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e [term 1 LEADER]: Becoming Leader. State: Replica: cd5fdb779ad846ad8afeb064ab59999e, State: Running, Role: LEADER
I20260808 13:08:43.762046 31918 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd5fdb779ad846ad8afeb064ab59999e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41517 } }
I20260808 13:08:43.762656 31906 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:08:43.764817 31922 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd5fdb779ad846ad8afeb064ab59999e. Latest consensus state: current_term: 1 leader_uuid: "cd5fdb779ad846ad8afeb064ab59999e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd5fdb779ad846ad8afeb064ab59999e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41517 } } }
I20260808 13:08:43.765235 31922 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e [sys.catalog]: This master's current role is: LEADER
I20260808 13:08:43.765233 31921 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd5fdb779ad846ad8afeb064ab59999e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd5fdb779ad846ad8afeb064ab59999e" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41517 } } }
I20260808 13:08:43.766183 31921 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e [sys.catalog]: This master's current role is: LEADER
I20260808 13:08:43.766335 31927 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:08:43.771823 31927 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:08:43.782393 31927 catalog_manager.cc:1347] Generated new cluster ID: 39739979b9ae44439b8de88a8a03119b
I20260808 13:08:43.782516 31927 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:08:43.802294 31927 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:08:43.804173 31927 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:08:43.811803 31927 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e: Generated new TSK 0
I20260808 13:08:43.812819 31927 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.1.124.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
Entry for principal kudu/127.1.124.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab.
I20260808 13:08:43.896359 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:41517
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:44293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:08:44.262598 31945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:44.262918 31945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:44.263091 31945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:44.270591 31945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:44.270756 31945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:44.270941 31945 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:08:44.281551 31945 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:44293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.129.keytab
--principal=kudu/127.1.124.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:41517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:44.282998 31945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:44.284425 31945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:44.298089 31951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:44.298166 31950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:44.298139 31953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:44.299182 31945 server_base.cc:1034] running on GCE node
I20260808 13:08:44.299764 31945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:44.300843 31945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:44.302103 31945 hybrid_clock.cc:648] HybridClock initialized: now 1786194524302034 us; error 58 us; skew 500 ppm
I20260808 13:08:44.302520 31945 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:08:44 dist-test-slave-t3q3 krb5kdc[31791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194524, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:08:44.305811 31945 init.cc:377] Logged in from keytab as kudu/127.1.124.129@KRBTEST.COM (short username kudu)
I20260808 13:08:44.307359 31945 webserver.cc:460] Webserver started at http://127.1.124.129:41093/ using document root <none> and password file <none>
I20260808 13:08:44.308126 31945 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:44.308266 31945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:44.308601 31945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:44.311573 31945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "11258cfca2d54c01ad2c891efbe1e5b0"
format_stamp: "Formatted at 2026-08-08 13:08:44 on dist-test-slave-t3q3"
server_key: "15bc36423e24097962c7e5e565c5bf8d"
server_key_iv: "7d2da10ae7aaf58651a3e24c2be0a278"
server_key_version: "encryptionkey@0"
I20260808 13:08:44.312491 31945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "11258cfca2d54c01ad2c891efbe1e5b0"
format_stamp: "Formatted at 2026-08-08 13:08:44 on dist-test-slave-t3q3"
server_key: "15bc36423e24097962c7e5e565c5bf8d"
server_key_iv: "7d2da10ae7aaf58651a3e24c2be0a278"
server_key_version: "encryptionkey@0"
I20260808 13:08:44.317922 31945 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:08:44.321527 31963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:44.322908 31945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260808 13:08:44.323143 31945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "11258cfca2d54c01ad2c891efbe1e5b0"
format_stamp: "Formatted at 2026-08-08 13:08:44 on dist-test-slave-t3q3"
server_key: "15bc36423e24097962c7e5e565c5bf8d"
server_key_iv: "7d2da10ae7aaf58651a3e24c2be0a278"
server_key_version: "encryptionkey@0"
I20260808 13:08:44.323355 31945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:44.335605 31945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:44.339646 31945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:44.339952 31945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:44.340938 31945 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:08:44.342312 31945 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:08:44.342424 31945 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:44.342581 31945 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:08:44.342654 31945 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:44.378759 31945 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:41491
I20260808 13:08:44.378813 32128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:41491 every 8 connection(s)
I20260808 13:08:44.380208 31945 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:08:44.381458 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 31945
I20260808 13:08:44.381683 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:08:44.382227 1522 external_mini_cluster.cc:1428] Setting key 3f961c68140e235348edcfcf4fef95a7
Aug 08 13:08:44 dist-test-slave-t3q3 krb5kdc[31791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194524, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.129@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:08:44.406764 32131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:44.383233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:36897 (local address 127.1.124.190:41517)
0808 13:08:44.383725 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:44.383729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:44.384942 (+ 1213us) server_negotiation.cc:407] Connection header received
0808 13:08:44.387957 (+ 3015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:44.387961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:44.388029 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:44.388220 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:44.390373 (+ 2153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:44.391126 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:44.393734 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:44.394186 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:44.394291 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:44.399182 (+ 4891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:44.399188 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:44.399191 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:44.399228 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:44.402893 (+ 3665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:44.403740 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:44.403749 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:44.403755 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:44.403926 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:44.404493 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:44.404501 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:44.404505 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:44.404939 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:44.405115 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:44.405958 (+ 843us) server_negotiation.cc:299] Negotiation successful
0808 13:08:44.406251 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":264,"threads_started":1}
I20260808 13:08:44.407990 32132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:44.383835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:41517 (local address 127.1.124.129:36897)
0808 13:08:44.384756 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:44.384807 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:44.387592 (+ 2785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:44.388479 (+ 887us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:44.388493 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:44.389310 (+ 817us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:08:44.390156 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:44.390187 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:44.391294 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:44.391302 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:44.393561 (+ 2259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:44.393580 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:44.394445 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:44.394455 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:44.394810 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:44.395615 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:08:44.395650 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:08:44.398876 (+ 3226us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:08:44.403059 (+ 4183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:08:44.403067 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:08:44.403083 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:08:44.403578 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:08:44.404132 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:08:44.404144 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:08:44.404147 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:08:44.404357 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:08:44.405136 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:08:44.405144 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:08:44.405510 (+ 366us) client_negotiation.cc:769] Sending connection context
0808 13:08:44.405804 (+ 294us) client_negotiation.cc:240] Negotiation successful
0808 13:08:44.406255 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":392,"threads_started":1}
I20260808 13:08:44.409942 32129 heartbeater.cc:347] Connected to a master server at 127.1.124.190:41517
I20260808 13:08:44.410319 32129 heartbeater.cc:464] Registering TS with master...
I20260808 13:08:44.411077 32129 heartbeater.cc:511] Master 127.1.124.190:41517 requested a full tablet report, sending...
I20260808 13:08:44.413661 31857 ts_manager.cc:194] Registered new tserver with Master: 11258cfca2d54c01ad2c891efbe1e5b0 (127.1.124.129:41491)
I20260808 13:08:44.416036 31857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.129@KRBTEST.COM'} at 127.1.124.129:36897
WARNING: no policy specified for kudu/127.1.124.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
Entry for principal kudu/127.1.124.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab.
I20260808 13:08:44.463443 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:41517
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:44293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:08:44.818532 32135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:44.818812 32135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:44.818997 32135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:44.826357 32135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:44.826475 32135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:44.826647 32135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:08:44.836647 32135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:44293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.130.keytab
--principal=kudu/127.1.124.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:41517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:44.838054 32135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:44.839438 32135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:44.853308 32142 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:44.853412 32143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:44.853336 32146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:44.857650 32135 server_base.cc:1034] running on GCE node
I20260808 13:08:44.858376 32135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:44.859338 32135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:44.860544 32135 hybrid_clock.cc:648] HybridClock initialized: now 1786194524860501 us; error 33 us; skew 500 ppm
I20260808 13:08:44.860846 32135 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:08:44 dist-test-slave-t3q3 krb5kdc[31791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194524, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:08:44.864297 32135 init.cc:377] Logged in from keytab as kudu/127.1.124.130@KRBTEST.COM (short username kudu)
I20260808 13:08:44.865962 32135 webserver.cc:460] Webserver started at http://127.1.124.130:36583/ using document root <none> and password file <none>
I20260808 13:08:44.866729 32135 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:44.866839 32135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:44.867216 32135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:44.870288 32135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "1b44229f93da40638689f03615cc34fd"
format_stamp: "Formatted at 2026-08-08 13:08:44 on dist-test-slave-t3q3"
server_key: "fcfcdce3b8f5b98343487a7846c5175b"
server_key_iv: "1db3bcf40361bdc508fa077f1f06bb18"
server_key_version: "encryptionkey@0"
I20260808 13:08:44.871104 32135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "1b44229f93da40638689f03615cc34fd"
format_stamp: "Formatted at 2026-08-08 13:08:44 on dist-test-slave-t3q3"
server_key: "fcfcdce3b8f5b98343487a7846c5175b"
server_key_iv: "1db3bcf40361bdc508fa077f1f06bb18"
server_key_version: "encryptionkey@0"
I20260808 13:08:44.876614 32135 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 13:08:44.880192 32154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:44.881528 32135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:08:44.881713 32135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "1b44229f93da40638689f03615cc34fd"
format_stamp: "Formatted at 2026-08-08 13:08:44 on dist-test-slave-t3q3"
server_key: "fcfcdce3b8f5b98343487a7846c5175b"
server_key_iv: "1db3bcf40361bdc508fa077f1f06bb18"
server_key_version: "encryptionkey@0"
I20260808 13:08:44.881955 32135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:44.903287 32135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:44.907689 32135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:44.907990 32135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:44.909025 32135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:08:44.910351 32135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:08:44.910429 32135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:44.910591 32135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:08:44.910660 32135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:44.944780 32135 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:38379
I20260808 13:08:44.944890 32329 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:38379 every 8 connection(s)
I20260808 13:08:44.946362 32135 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:08:44.946748 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 32135
I20260808 13:08:44.946954 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:08:44.947567 1522 external_mini_cluster.cc:1428] Setting key d6d6f6c992df93a9696250526cef3d71
Aug 08 13:08:44 dist-test-slave-t3q3 krb5kdc[31791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194524, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.130@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:08:44.970645 32336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:44.949358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:33307 (local address 127.1.124.190:41517)
0808 13:08:44.949832 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:44.949838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:44.951159 (+ 1321us) server_negotiation.cc:407] Connection header received
0808 13:08:44.954171 (+ 3012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:44.954176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:44.954240 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:44.954335 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:44.956493 (+ 2158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:44.957268 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:44.959778 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:44.960276 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:44.960378 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:44.964732 (+ 4354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:44.964737 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:44.964740 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:44.964771 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:44.967398 (+ 2627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:44.968268 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:44.968275 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:44.968283 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:44.968413 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:44.968869 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:44.968874 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:44.968876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:44.969155 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:44.969312 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:44.970025 (+ 713us) server_negotiation.cc:299] Negotiation successful
0808 13:08:44.970259 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":256,"threads_started":1}
I20260808 13:08:44.971657 32338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:44.950054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:41517 (local address 127.1.124.130:33307)
0808 13:08:44.950967 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:44.951018 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:44.953862 (+ 2844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:44.954632 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:44.954645 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:44.955425 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:08:44.956287 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:44.956318 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:44.957427 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:44.957435 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:44.959611 (+ 2176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:44.959630 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:44.960502 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:44.960511 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:44.960860 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:44.961606 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:08:44.961640 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:08:44.964481 (+ 2841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:08:44.967601 (+ 3120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:08:44.967610 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:08:44.967626 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:08:44.968132 (+ 506us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:08:44.968536 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:08:44.968541 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:08:44.968544 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:08:44.968737 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:08:44.969288 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:08:44.969296 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:08:44.969642 (+ 346us) client_negotiation.cc:769] Sending connection context
0808 13:08:44.969906 (+ 264us) client_negotiation.cc:240] Negotiation successful
0808 13:08:44.970441 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":591,"thread_start_us":303,"threads_started":1}
I20260808 13:08:44.973217 32333 heartbeater.cc:347] Connected to a master server at 127.1.124.190:41517
I20260808 13:08:44.973565 32333 heartbeater.cc:464] Registering TS with master...
I20260808 13:08:44.974454 32333 heartbeater.cc:511] Master 127.1.124.190:41517 requested a full tablet report, sending...
I20260808 13:08:44.976166 31857 ts_manager.cc:194] Registered new tserver with Master: 1b44229f93da40638689f03615cc34fd (127.1.124.130:38379)
I20260808 13:08:44.976989 31857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.130@KRBTEST.COM'} at 127.1.124.130:33307
WARNING: no policy specified for kudu/127.1.124.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
Entry for principal kudu/127.1.124.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab.
I20260808 13:08:45.025055 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:41517
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:44293
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:08:45.384855 32343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:45.385255 32343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:45.385485 32343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:45.393953 32343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:45.394081 32343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:45.394210 32343 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:08:45.404824 32343 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:44293
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.131.keytab
--principal=kudu/127.1.124.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:41517
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:45.406141 32343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:45.407390 32343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 13:08:45.419644 32129 heartbeater.cc:503] Master 127.1.124.190:41517 was elected leader, sending a full tablet report...
W20260808 13:08:45.421028 32352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:45.421000 32354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:45.421000 32351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:45.422232 32343 server_base.cc:1034] running on GCE node
I20260808 13:08:45.422851 32343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:45.423844 32343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:45.425112 32343 hybrid_clock.cc:648] HybridClock initialized: now 1786194525425033 us; error 50 us; skew 500 ppm
I20260808 13:08:45.425457 32343 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:08:45 dist-test-slave-t3q3 krb5kdc[31791](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194525, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:08:45.429258 32343 init.cc:377] Logged in from keytab as kudu/127.1.124.131@KRBTEST.COM (short username kudu)
I20260808 13:08:45.430716 32343 webserver.cc:460] Webserver started at http://127.1.124.131:35583/ using document root <none> and password file <none>
I20260808 13:08:45.431433 32343 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:45.431594 32343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:45.431924 32343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:45.434983 32343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "c0bd897973c7407fb947979b827975c3"
format_stamp: "Formatted at 2026-08-08 13:08:45 on dist-test-slave-t3q3"
server_key: "cf49706b7917ce8673b19047608de194"
server_key_iv: "bfeea0640cfeace2c4e6ea08c9b31e47"
server_key_version: "encryptionkey@0"
I20260808 13:08:45.435828 32343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "c0bd897973c7407fb947979b827975c3"
format_stamp: "Formatted at 2026-08-08 13:08:45 on dist-test-slave-t3q3"
server_key: "cf49706b7917ce8673b19047608de194"
server_key_iv: "bfeea0640cfeace2c4e6ea08c9b31e47"
server_key_version: "encryptionkey@0"
I20260808 13:08:45.441423 32343 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 13:08:45.445003 32366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:45.446367 32343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:08:45.446552 32343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "c0bd897973c7407fb947979b827975c3"
format_stamp: "Formatted at 2026-08-08 13:08:45 on dist-test-slave-t3q3"
server_key: "cf49706b7917ce8673b19047608de194"
server_key_iv: "bfeea0640cfeace2c4e6ea08c9b31e47"
server_key_version: "encryptionkey@0"
I20260808 13:08:45.446765 32343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:45.464864 32343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:45.469197 32343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:45.469506 32343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:45.470521 32343 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:08:45.472330 32343 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:08:45.472455 32343 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:45.472602 32343 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:08:45.472684 32343 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:45.506317 32343 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:32955
I20260808 13:08:45.506364 32529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:32955 every 8 connection(s)
I20260808 13:08:45.507685 32343 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:08:45.508589 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 32343
I20260808 13:08:45.508792 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:08:45.509364 1522 external_mini_cluster.cc:1428] Setting key e5635a41533de4ac599bba6d4aa7cbbe
Aug 08 13:08:45 dist-test-slave-t3q3 krb5kdc[31791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194525, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.131@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:08:45.529711 32534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:45.510700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:43853 (local address 127.1.124.190:41517)
0808 13:08:45.511198 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:45.511207 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:45.512567 (+ 1360us) server_negotiation.cc:407] Connection header received
0808 13:08:45.514407 (+ 1840us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:45.514411 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:45.514488 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:45.514587 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:45.516041 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.516907 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.518749 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.519175 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.519294 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:45.523198 (+ 3904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:45.523203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:45.523206 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:45.523246 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:45.526549 (+ 3303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:45.527356 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:45.527360 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:45.527362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:45.527482 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:45.527916 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:45.527921 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:45.527923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:45.528358 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:45.528497 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:45.529257 (+ 760us) server_negotiation.cc:299] Negotiation successful
0808 13:08:45.529430 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":222,"threads_started":1}
I20260808 13:08:45.530740 32535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:45.511402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:41517 (local address 127.1.124.131:43853)
0808 13:08:45.512389 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:45.512448 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:45.514158 (+ 1710us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:45.514789 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:45.514797 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:45.515344 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:08:45.515887 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:45.515904 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.517135 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.517140 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:45.518580 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:45.518594 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.519416 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.519422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:45.519687 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:45.520478 (+ 791us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:08:45.520507 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:08:45.522933 (+ 2426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:08:45.526678 (+ 3745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:08:45.526693 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:08:45.526708 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:08:45.527237 (+ 529us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:08:45.527578 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:08:45.527581 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:08:45.527584 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:08:45.527769 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:08:45.528502 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:08:45.528509 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:08:45.528873 (+ 364us) client_negotiation.cc:769] Sending connection context
0808 13:08:45.529167 (+ 294us) client_negotiation.cc:240] Negotiation successful
0808 13:08:45.529584 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"thread_start_us":352,"threads_started":1}
I20260808 13:08:45.532234 32530 heartbeater.cc:347] Connected to a master server at 127.1.124.190:41517
I20260808 13:08:45.532632 32530 heartbeater.cc:464] Registering TS with master...
I20260808 13:08:45.533663 32530 heartbeater.cc:511] Master 127.1.124.190:41517 requested a full tablet report, sending...
I20260808 13:08:45.535117 31857 ts_manager.cc:194] Registered new tserver with Master: c0bd897973c7407fb947979b827975c3 (127.1.124.131:32955)
I20260808 13:08:45.535893 31857 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.1.124.131@KRBTEST.COM'} at 127.1.124.131:43853
I20260808 13:08:45.538759 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:08:45.556470 32534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:45.542935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48784 (local address 127.1.124.190:41517)
0808 13:08:45.543154 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:45.543163 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:45.543645 (+ 482us) server_negotiation.cc:407] Connection header received
0808 13:08:45.543787 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:45.543791 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:45.543855 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:45.543939 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:45.545472 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.546286 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.547670 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.548158 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.548261 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:45.550954 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:45.550960 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:45.550963 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:45.550996 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:45.553774 (+ 2778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:45.554479 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:45.554484 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:45.554500 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:45.554634 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:45.554991 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:45.554995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:45.554997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:45.555333 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:45.555546 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:45.555851 (+ 305us) server_negotiation.cc:299] Negotiation successful
0808 13:08:45.556046 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 13:08:45.560739 31857 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48784:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:08:45.567826 31857 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:08:45.594129 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:45.582996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41146 (local address 127.1.124.130:38379)
0808 13:08:45.584765 (+ 1769us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:45.584777 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:45.584810 (+ 33us) server_negotiation.cc:407] Connection header received
0808 13:08:45.584916 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:45.584924 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:45.585159 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:45.586506 (+ 1347us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:45.586671 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.587944 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.591483 (+ 3539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.592729 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.592870 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:45.593102 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:45.593659 (+ 557us) server_negotiation.cc:299] Negotiation successful
0808 13:08:45.593822 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1580,"thread_start_us":1460,"threads_started":1}
I20260808 13:08:45.594219 32550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:45.581364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:41491 (local address 127.0.0.1:35332)
0808 13:08:45.582581 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:45.582634 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:45.582851 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:45.585269 (+ 2418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:45.585277 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:45.585333 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:45.585875 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:45.585891 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.588226 (+ 2335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.588231 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:45.590922 (+ 2691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:45.590942 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.593000 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.593009 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:45.593248 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:45.593303 (+ 55us) client_negotiation.cc:769] Sending connection context
0808 13:08:45.593438 (+ 135us) client_negotiation.cc:240] Negotiation successful
0808 13:08:45.593666 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":886,"mutex_wait_us":110,"spinlock_wait_cycles":5504,"thread_start_us":633,"threads_started":1}
I20260808 13:08:45.594265 32553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:45.582329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:38379 (local address 127.0.0.1:41146)
0808 13:08:45.583498 (+ 1169us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:45.583525 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:45.583697 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:45.585436 (+ 1739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:45.585440 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:45.585467 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:45.586284 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:45.586296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.588097 (+ 1801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.588103 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:45.591328 (+ 3225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:45.591339 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.593262 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.593270 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:45.593445 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:45.593480 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 13:08:45.593605 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 13:08:45.593738 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":997,"mutex_wait_us":513,"thread_start_us":855,"threads_started":1}
I20260808 13:08:45.595031 32555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:45.583518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35590 (local address 127.1.124.131:32955)
0808 13:08:45.586116 (+ 2598us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:45.586125 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:45.586160 (+ 35us) server_negotiation.cc:407] Connection header received
0808 13:08:45.586340 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:45.586348 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:45.586579 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:45.586870 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:45.587372 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.589156 (+ 1784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.591577 (+ 2421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.593027 (+ 1450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.593170 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:45.593405 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:45.594475 (+ 1070us) server_negotiation.cc:299] Negotiation successful
0808 13:08:45.594655 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2332,"spinlock_wait_cycles":25472,"thread_start_us":2297,"threads_started":1}
I20260808 13:08:45.595477 32556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:45.584028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:32955 (local address 127.0.0.1:35590)
0808 13:08:45.585184 (+ 1156us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:45.585204 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:45.585309 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:45.586801 (+ 1492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:45.586808 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:45.586835 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:45.587176 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:45.587188 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.589338 (+ 2150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.589343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:45.591377 (+ 2034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:45.591393 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.593584 (+ 2191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.593590 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:45.593743 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:45.593782 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 13:08:45.593885 (+ 103us) client_negotiation.cc:240] Negotiation successful
0808 13:08:45.594009 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":954,"mutex_wait_us":137,"thread_start_us":746,"threads_started":1}
I20260808 13:08:45.595763 32551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:45.581748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35332 (local address 127.1.124.129:41491)
0808 13:08:45.584516 (+ 2768us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:45.584527 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:45.584561 (+ 34us) server_negotiation.cc:407] Connection header received
0808 13:08:45.584670 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:45.584682 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:45.584992 (+ 310us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:45.585775 (+ 783us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:45.586032 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.588042 (+ 2010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.591120 (+ 3078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.592712 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.594926 (+ 2214us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:45.595172 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:45.595246 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 13:08:45.595411 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2508,"thread_start_us":2298,"threads_started":1}
I20260808 13:08:45.599164 32232 tablet_service.cc:1467] Processing CreateTablet for tablet ad93d823cdb14efa8ff138ff12baa254 (DEFAULT_TABLE table=test-table [id=69594f08967949b4bcc607964d285cf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:08:45.599534 32440 tablet_service.cc:1467] Processing CreateTablet for tablet ad93d823cdb14efa8ff138ff12baa254 (DEFAULT_TABLE table=test-table [id=69594f08967949b4bcc607964d285cf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:08:45.599532 32038 tablet_service.cc:1467] Processing CreateTablet for tablet ad93d823cdb14efa8ff138ff12baa254 (DEFAULT_TABLE table=test-table [id=69594f08967949b4bcc607964d285cf4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:08:45.600569 32232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad93d823cdb14efa8ff138ff12baa254. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:45.601028 32440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad93d823cdb14efa8ff138ff12baa254. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:45.601028 32038 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ad93d823cdb14efa8ff138ff12baa254. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:45.612634 32558 tablet_bootstrap.cc:492] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd: Bootstrap starting.
I20260808 13:08:45.616691 32558 tablet_bootstrap.cc:654] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:45.617168 32562 tablet_bootstrap.cc:492] T ad93d823cdb14efa8ff138ff12baa254 P c0bd897973c7407fb947979b827975c3: Bootstrap starting.
I20260808 13:08:45.618336 32558 log.cc:824] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:45.619058 32561 tablet_bootstrap.cc:492] T ad93d823cdb14efa8ff138ff12baa254 P 11258cfca2d54c01ad2c891efbe1e5b0: Bootstrap starting.
I20260808 13:08:45.621122 32562 tablet_bootstrap.cc:654] T ad93d823cdb14efa8ff138ff12baa254 P c0bd897973c7407fb947979b827975c3: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:45.622547 32561 tablet_bootstrap.cc:654] T ad93d823cdb14efa8ff138ff12baa254 P 11258cfca2d54c01ad2c891efbe1e5b0: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:45.622535 32562 log.cc:824] T ad93d823cdb14efa8ff138ff12baa254 P c0bd897973c7407fb947979b827975c3: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:45.623322 32558 tablet_bootstrap.cc:492] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd: No bootstrap required, opened a new log
I20260808 13:08:45.623615 32558 ts_tablet_manager.cc:1397] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.003s
I20260808 13:08:45.623811 32561 log.cc:824] T ad93d823cdb14efa8ff138ff12baa254 P 11258cfca2d54c01ad2c891efbe1e5b0: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:45.627171 32562 tablet_bootstrap.cc:492] T ad93d823cdb14efa8ff138ff12baa254 P c0bd897973c7407fb947979b827975c3: No bootstrap required, opened a new log
I20260808 13:08:45.627177 32561 tablet_bootstrap.cc:492] T ad93d823cdb14efa8ff138ff12baa254 P 11258cfca2d54c01ad2c891efbe1e5b0: No bootstrap required, opened a new log
I20260808 13:08:45.627475 32562 ts_tablet_manager.cc:1397] T ad93d823cdb14efa8ff138ff12baa254 P c0bd897973c7407fb947979b827975c3: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260808 13:08:45.627480 32561 ts_tablet_manager.cc:1397] T ad93d823cdb14efa8ff138ff12baa254 P 11258cfca2d54c01ad2c891efbe1e5b0: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260808 13:08:45.631989 32558 raft_consensus.cc:348] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11258cfca2d54c01ad2c891efbe1e5b0" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41491 } } peers { permanent_uuid: "c0bd897973c7407fb947979b827975c3" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 32955 } } peers { permanent_uuid: "1b44229f93da40638689f03615cc34fd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 38379 } }
I20260808 13:08:45.632501 32558 raft_consensus.cc:374] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:45.632613 32558 raft_consensus.cc:729] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b44229f93da40638689f03615cc34fd, State: Initialized, Role: FOLLOWER
I20260808 13:08:45.633327 32558 consensus_queue.cc:260] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11258cfca2d54c01ad2c891efbe1e5b0" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41491 } } peers { permanent_uuid: "c0bd897973c7407fb947979b827975c3" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 32955 } } peers { permanent_uuid: "1b44229f93da40638689f03615cc34fd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 38379 } }
I20260808 13:08:45.634542 32333 heartbeater.cc:503] Master 127.1.124.190:41517 was elected leader, sending a full tablet report...
I20260808 13:08:45.635313 32558 ts_tablet_manager.cc:1428] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260808 13:08:45.639030 32561 raft_consensus.cc:348] T ad93d823cdb14efa8ff138ff12baa254 P 11258cfca2d54c01ad2c891efbe1e5b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11258cfca2d54c01ad2c891efbe1e5b0" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41491 } } peers { permanent_uuid: "c0bd897973c7407fb947979b827975c3" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 32955 } } peers { permanent_uuid: "1b44229f93da40638689f03615cc34fd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 38379 } }
I20260808 13:08:45.639030 32562 raft_consensus.cc:348] T ad93d823cdb14efa8ff138ff12baa254 P c0bd897973c7407fb947979b827975c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11258cfca2d54c01ad2c891efbe1e5b0" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41491 } } peers { permanent_uuid: "c0bd897973c7407fb947979b827975c3" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 32955 } } peers { permanent_uuid: "1b44229f93da40638689f03615cc34fd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 38379 } }
I20260808 13:08:45.639556 32561 raft_consensus.cc:374] T ad93d823cdb14efa8ff138ff12baa254 P 11258cfca2d54c01ad2c891efbe1e5b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:45.639547 32562 raft_consensus.cc:374] T ad93d823cdb14efa8ff138ff12baa254 P c0bd897973c7407fb947979b827975c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:45.639672 32562 raft_consensus.cc:729] T ad93d823cdb14efa8ff138ff12baa254 P c0bd897973c7407fb947979b827975c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c0bd897973c7407fb947979b827975c3, State: Initialized, Role: FOLLOWER
I20260808 13:08:45.639667 32561 raft_consensus.cc:729] T ad93d823cdb14efa8ff138ff12baa254 P 11258cfca2d54c01ad2c891efbe1e5b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11258cfca2d54c01ad2c891efbe1e5b0, State: Initialized, Role: FOLLOWER
I20260808 13:08:45.640503 32561 consensus_queue.cc:260] T ad93d823cdb14efa8ff138ff12baa254 P 11258cfca2d54c01ad2c891efbe1e5b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11258cfca2d54c01ad2c891efbe1e5b0" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41491 } } peers { permanent_uuid: "c0bd897973c7407fb947979b827975c3" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 32955 } } peers { permanent_uuid: "1b44229f93da40638689f03615cc34fd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 38379 } }
I20260808 13:08:45.640503 32562 consensus_queue.cc:260] T ad93d823cdb14efa8ff138ff12baa254 P c0bd897973c7407fb947979b827975c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11258cfca2d54c01ad2c891efbe1e5b0" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41491 } } peers { permanent_uuid: "c0bd897973c7407fb947979b827975c3" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 32955 } } peers { permanent_uuid: "1b44229f93da40638689f03615cc34fd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 38379 } }
I20260808 13:08:45.642326 32530 heartbeater.cc:503] Master 127.1.124.190:41517 was elected leader, sending a full tablet report...
I20260808 13:08:45.642715 32561 ts_tablet_manager.cc:1428] T ad93d823cdb14efa8ff138ff12baa254 P 11258cfca2d54c01ad2c891efbe1e5b0: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260808 13:08:45.642943 32562 ts_tablet_manager.cc:1428] T ad93d823cdb14efa8ff138ff12baa254 P c0bd897973c7407fb947979b827975c3: Time spent starting tablet: real 0.015s user 0.011s sys 0.006s
W20260808 13:08:45.698729 32335 tablet.cc:2369] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:08:45.742007 32567 raft_consensus.cc:482] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:08:45.742491 32567 raft_consensus.cc:504] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11258cfca2d54c01ad2c891efbe1e5b0" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41491 } } peers { permanent_uuid: "c0bd897973c7407fb947979b827975c3" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 32955 } } peers { permanent_uuid: "1b44229f93da40638689f03615cc34fd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 38379 } }
I20260808 13:08:45.744362 32567 leader_election.cc:288] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 11258cfca2d54c01ad2c891efbe1e5b0 (127.1.124.129:41491), c0bd897973c7407fb947979b827975c3 (127.1.124.131:32955)
I20260808 13:08:45.752034 32551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:45.744896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:38071 (local address 127.1.124.129:41491)
0808 13:08:45.745180 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:45.745190 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:45.745668 (+ 478us) server_negotiation.cc:407] Connection header received
0808 13:08:45.745909 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:45.745918 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:45.746016 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:45.746117 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:45.746985 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.748003 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.750018 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.751000 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.751126 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:45.751215 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:45.751616 (+ 401us) server_negotiation.cc:299] Negotiation successful
0808 13:08:45.751758 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260808 13:08:45.752202 32571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:45.744745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:41491 (local address 127.1.124.130:38071)
0808 13:08:45.745516 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:45.745533 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:45.745717 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:45.746295 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:45.746300 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:45.746333 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:45.746815 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:45.746831 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.748179 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.748184 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:45.749853 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:45.749865 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.751223 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.751229 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:45.751413 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:45.751453 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 13:08:45.751567 (+ 114us) client_negotiation.cc:240] Negotiation successful
0808 13:08:45.751706 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"spinlock_wait_cycles":27904,"thread_start_us":499,"threads_started":1}
I20260808 13:08:45.752691 32555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:45.746023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:40091 (local address 127.1.124.131:32955)
0808 13:08:45.746261 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:45.746267 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:45.746496 (+ 229us) server_negotiation.cc:407] Connection header received
0808 13:08:45.746660 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:45.746668 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:45.746755 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:45.746871 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:45.747615 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.748586 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.750474 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.751530 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.751690 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:45.751754 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:45.752292 (+ 538us) server_negotiation.cc:299] Negotiation successful
0808 13:08:45.752436 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 13:08:45.753362 32575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:45.745695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:32955 (local address 127.1.124.130:40091)
0808 13:08:45.746389 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:45.746403 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:45.746519 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:45.746957 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:45.746962 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:45.746988 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:45.747428 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:45.747440 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.748736 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.748740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:45.750311 (+ 1571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:45.750326 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.751778 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.751784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:45.752000 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:45.752041 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 13:08:45.752873 (+ 832us) client_negotiation.cc:240] Negotiation successful
0808 13:08:45.753029 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":295,"threads_started":1}
I20260808 13:08:45.753399 32066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad93d823cdb14efa8ff138ff12baa254" candidate_uuid: "1b44229f93da40638689f03615cc34fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11258cfca2d54c01ad2c891efbe1e5b0" is_pre_election: true
I20260808 13:08:45.754040 32066 raft_consensus.cc:2454] T ad93d823cdb14efa8ff138ff12baa254 P 11258cfca2d54c01ad2c891efbe1e5b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b44229f93da40638689f03615cc34fd in term 0.
I20260808 13:08:45.754446 32469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad93d823cdb14efa8ff138ff12baa254" candidate_uuid: "1b44229f93da40638689f03615cc34fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0bd897973c7407fb947979b827975c3" is_pre_election: true
I20260808 13:08:45.754805 32159 leader_election.cc:302] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11258cfca2d54c01ad2c891efbe1e5b0, 1b44229f93da40638689f03615cc34fd; no voters:
I20260808 13:08:45.754925 32469 raft_consensus.cc:2454] T ad93d823cdb14efa8ff138ff12baa254 P c0bd897973c7407fb947979b827975c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1b44229f93da40638689f03615cc34fd in term 0.
I20260808 13:08:45.755236 32567 raft_consensus.cc:2781] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:08:45.755421 32567 raft_consensus.cc:482] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:08:45.755540 32567 raft_consensus.cc:3037] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:45.757835 32567 raft_consensus.cc:504] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11258cfca2d54c01ad2c891efbe1e5b0" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41491 } } peers { permanent_uuid: "c0bd897973c7407fb947979b827975c3" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 32955 } } peers { permanent_uuid: "1b44229f93da40638689f03615cc34fd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 38379 } }
I20260808 13:08:45.758440 32567 leader_election.cc:288] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd [CANDIDATE]: Term 1 election: Requested vote from peers 11258cfca2d54c01ad2c891efbe1e5b0 (127.1.124.129:41491), c0bd897973c7407fb947979b827975c3 (127.1.124.131:32955)
I20260808 13:08:45.758910 32066 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad93d823cdb14efa8ff138ff12baa254" candidate_uuid: "1b44229f93da40638689f03615cc34fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "11258cfca2d54c01ad2c891efbe1e5b0"
I20260808 13:08:45.758968 32469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ad93d823cdb14efa8ff138ff12baa254" candidate_uuid: "1b44229f93da40638689f03615cc34fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c0bd897973c7407fb947979b827975c3"
I20260808 13:08:45.759099 32066 raft_consensus.cc:3037] T ad93d823cdb14efa8ff138ff12baa254 P 11258cfca2d54c01ad2c891efbe1e5b0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:45.759253 32469 raft_consensus.cc:3037] T ad93d823cdb14efa8ff138ff12baa254 P c0bd897973c7407fb947979b827975c3 [term 0 FOLLOWER]: Advancing to term 1
W20260808 13:08:45.760150 32532 tablet.cc:2369] T ad93d823cdb14efa8ff138ff12baa254 P c0bd897973c7407fb947979b827975c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:08:45.761499 32066 raft_consensus.cc:2454] T ad93d823cdb14efa8ff138ff12baa254 P 11258cfca2d54c01ad2c891efbe1e5b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b44229f93da40638689f03615cc34fd in term 1.
I20260808 13:08:45.761904 32159 leader_election.cc:302] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11258cfca2d54c01ad2c891efbe1e5b0, 1b44229f93da40638689f03615cc34fd; no voters:
I20260808 13:08:45.762202 32567 raft_consensus.cc:2781] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:08:45.762743 32567 raft_consensus.cc:686] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd [term 1 LEADER]: Becoming Leader. State: Replica: 1b44229f93da40638689f03615cc34fd, State: Running, Role: LEADER
I20260808 13:08:45.762799 32469 raft_consensus.cc:2454] T ad93d823cdb14efa8ff138ff12baa254 P c0bd897973c7407fb947979b827975c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1b44229f93da40638689f03615cc34fd in term 1.
I20260808 13:08:45.763631 32567 consensus_queue.cc:237] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11258cfca2d54c01ad2c891efbe1e5b0" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41491 } } peers { permanent_uuid: "c0bd897973c7407fb947979b827975c3" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 32955 } } peers { permanent_uuid: "1b44229f93da40638689f03615cc34fd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 38379 } }
I20260808 13:08:45.768447 31857 catalog_manager.cc:5495] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd reported cstate change: term changed from 0 to 1, leader changed from <none> to 1b44229f93da40638689f03615cc34fd (127.1.124.130). New cstate: current_term: 1 leader_uuid: "1b44229f93da40638689f03615cc34fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11258cfca2d54c01ad2c891efbe1e5b0" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41491 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c0bd897973c7407fb947979b827975c3" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 32955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b44229f93da40638689f03615cc34fd" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 38379 } health_report { overall_health: HEALTHY } } }
I20260808 13:08:45.792128 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:45.786519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41158 (local address 127.1.124.130:38379)
0808 13:08:45.786789 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:45.786793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:45.786840 (+ 47us) server_negotiation.cc:407] Connection header received
0808 13:08:45.787073 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:45.787079 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:45.787169 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:45.787298 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:08:45.787702 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.788468 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.789905 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.790322 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.790424 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:45.790895 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:08:45.791411 (+ 516us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:08:45.791528 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:45.791638 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 13:08:45.791714 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 13:08:45.800621 32469 raft_consensus.cc:1261] T ad93d823cdb14efa8ff138ff12baa254 P c0bd897973c7407fb947979b827975c3 [term 1 FOLLOWER]: Refusing update from remote peer 1b44229f93da40638689f03615cc34fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:08:45.800621 32066 raft_consensus.cc:1261] T ad93d823cdb14efa8ff138ff12baa254 P 11258cfca2d54c01ad2c891efbe1e5b0 [term 1 FOLLOWER]: Refusing update from remote peer 1b44229f93da40638689f03615cc34fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:08:45.801697 32576 consensus_queue.cc:1035] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "c0bd897973c7407fb947979b827975c3" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 32955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:08:45.802112 32567 consensus_queue.cc:1035] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "11258cfca2d54c01ad2c891efbe1e5b0" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 41491 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:08:45.814312 32581 mvcc.cc:205] Tried to move back new op lower bound from 7316252777668358144 to 7316252777532338176. Current Snapshot: MvccSnapshot[applied={T|T < 7316252777668358144}]
I20260808 13:08:45.819244 32584 mvcc.cc:205] Tried to move back new op lower bound from 7316252777668358144 to 7316252777532338176. Current Snapshot: MvccSnapshot[applied={T|T < 7316252777668358144}]
I20260808 13:08:45.830305 31857 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:08:45.830574 31857 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48784:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:08:45.834543 31857 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cd5fdb779ad846ad8afeb064ab59999e: Sending DeleteTablet for 3 replicas of tablet ad93d823cdb14efa8ff138ff12baa254
I20260808 13:08:45.836026 32440 tablet_service.cc:1514] Processing DeleteTablet for tablet ad93d823cdb14efa8ff138ff12baa254 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:08:45 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:35590
I20260808 13:08:45.836095 32232 tablet_service.cc:1514] Processing DeleteTablet for tablet ad93d823cdb14efa8ff138ff12baa254 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:08:45 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:41146
I20260808 13:08:45.836024 32038 tablet_service.cc:1514] Processing DeleteTablet for tablet ad93d823cdb14efa8ff138ff12baa254 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:08:45 UTC) from {username='kudu', principal='kudu/127.1.124.190@KRBTEST.COM'} at 127.0.0.1:35332
I20260808 13:08:45.838522 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 31945
I20260808 13:08:45.848440 32598 tablet_replica.cc:330] T ad93d823cdb14efa8ff138ff12baa254 P c0bd897973c7407fb947979b827975c3: stopping tablet replica
I20260808 13:08:45.849283 32598 raft_consensus.cc:2229] T ad93d823cdb14efa8ff138ff12baa254 P c0bd897973c7407fb947979b827975c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:08:45.850220 32598 raft_consensus.cc:2258] T ad93d823cdb14efa8ff138ff12baa254 P c0bd897973c7407fb947979b827975c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:08:45.853250 32598 ts_tablet_manager.cc:1905] T ad93d823cdb14efa8ff138ff12baa254 P c0bd897973c7407fb947979b827975c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:08:45.860698 32596 tablet_replica.cc:330] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd: stopping tablet replica
I20260808 13:08:45.860881 32595 tablet_replica.cc:330] T ad93d823cdb14efa8ff138ff12baa254 P 11258cfca2d54c01ad2c891efbe1e5b0: stopping tablet replica
I20260808 13:08:45.861379 32596 raft_consensus.cc:2229] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:08:45.861383 32595 raft_consensus.cc:2229] T ad93d823cdb14efa8ff138ff12baa254 P 11258cfca2d54c01ad2c891efbe1e5b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:08:45.861963 32595 raft_consensus.cc:2258] T ad93d823cdb14efa8ff138ff12baa254 P 11258cfca2d54c01ad2c891efbe1e5b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:08:45.862340 32596 raft_consensus.cc:2258] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:08:45.863679 32598 ts_tablet_manager.cc:1918] T ad93d823cdb14efa8ff138ff12baa254 P c0bd897973c7407fb947979b827975c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:08:45.863862 32598 log.cc:1196] T ad93d823cdb14efa8ff138ff12baa254 P c0bd897973c7407fb947979b827975c3: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/ad93d823cdb14efa8ff138ff12baa254
I20260808 13:08:45.864516 32598 ts_tablet_manager.cc:1939] T ad93d823cdb14efa8ff138ff12baa254 P c0bd897973c7407fb947979b827975c3: Deleting consensus metadata
I20260808 13:08:45.864996 32595 ts_tablet_manager.cc:1905] T ad93d823cdb14efa8ff138ff12baa254 P 11258cfca2d54c01ad2c891efbe1e5b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:08:45.865298 32596 ts_tablet_manager.cc:1905] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:08:45.866480 31838 catalog_manager.cc:4841] TS c0bd897973c7407fb947979b827975c3 (127.1.124.131:32955): tablet ad93d823cdb14efa8ff138ff12baa254 (table test-table [id=69594f08967949b4bcc607964d285cf4]) successfully deleted
Aug 08 13:08:45 dist-test-slave-t3q3 krb5kdc[31791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.129@KRBTEST.COM
I20260808 13:08:45.872301 32595 ts_tablet_manager.cc:1918] T ad93d823cdb14efa8ff138ff12baa254 P 11258cfca2d54c01ad2c891efbe1e5b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:08:45.872517 32595 log.cc:1196] T ad93d823cdb14efa8ff138ff12baa254 P 11258cfca2d54c01ad2c891efbe1e5b0: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-0/wal/wals/ad93d823cdb14efa8ff138ff12baa254
I20260808 13:08:45.873204 32595 ts_tablet_manager.cc:1939] T ad93d823cdb14efa8ff138ff12baa254 P 11258cfca2d54c01ad2c891efbe1e5b0: Deleting consensus metadata
I20260808 13:08:45.873597 32596 ts_tablet_manager.cc:1918] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:08:45.873842 32596 log.cc:1196] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/ad93d823cdb14efa8ff138ff12baa254
I20260808 13:08:45.874469 32596 ts_tablet_manager.cc:1939] T ad93d823cdb14efa8ff138ff12baa254 P 1b44229f93da40638689f03615cc34fd: Deleting consensus metadata
I20260808 13:08:45.875365 31838 catalog_manager.cc:4841] TS 11258cfca2d54c01ad2c891efbe1e5b0 (127.1.124.129:41491): tablet ad93d823cdb14efa8ff138ff12baa254 (table test-table [id=69594f08967949b4bcc607964d285cf4]) successfully deleted
I20260808 13:08:45.876361 31839 catalog_manager.cc:4841] TS 1b44229f93da40638689f03615cc34fd (127.1.124.130:38379): tablet ad93d823cdb14efa8ff138ff12baa254 (table test-table [id=69594f08967949b4bcc607964d285cf4]) successfully deleted
I20260808 13:08:45.879536 32551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:45.840355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35346 (local address 127.1.124.129:41491)
0808 13:08:45.840596 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:45.840605 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:45.845351 (+ 4746us) server_negotiation.cc:407] Connection header received
0808 13:08:45.845572 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:45.845578 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:45.845690 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:45.845809 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:45.857252 (+ 11443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.858437 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.860766 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:45.861492 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:45.861608 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:45.871803 (+ 10195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:45.871819 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:45.871843 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:45.871895 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:45.876102 (+ 4207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:45.877098 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:45.877108 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:45.877112 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:45.877256 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:45.877795 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:45.877800 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:45.877803 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:45.878222 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:45.878498 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:45.878918 (+ 420us) server_negotiation.cc:299] Negotiation successful
0808 13:08:45.879110 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 13:08:45.880311 32124 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:45.973260 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 31945
I20260808 13:08:45.997541 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 32135
Aug 08 13:08:46 dist-test-slave-t3q3 krb5kdc[31791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.130@KRBTEST.COM
I20260808 13:08:46.014986 32554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:45.998409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41170 (local address 127.1.124.130:38379)
0808 13:08:45.998587 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:45.998593 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:45.998664 (+ 71us) server_negotiation.cc:407] Connection header received
0808 13:08:45.998879 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:45.998885 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:45.998980 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:45.999171 (+ 191us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:46.000581 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:46.001416 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:46.002742 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:46.003195 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:46.003348 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:46.008428 (+ 5080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:46.008451 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:46.008479 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:46.008561 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:46.011865 (+ 3304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:46.012768 (+ 903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:46.012775 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:46.012777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:46.012919 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:46.013351 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:46.013355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:46.013357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:46.013736 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:46.013999 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:46.014428 (+ 429us) server_negotiation.cc:299] Negotiation successful
0808 13:08:46.014623 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 13:08:46.015848 32325 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:46.131556 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 32135
I20260808 13:08:46.158543 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 32343
Aug 08 13:08:46 dist-test-slave-t3q3 krb5kdc[31791](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194523, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.131@KRBTEST.COM
I20260808 13:08:46.175304 32555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:46.159338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35600 (local address 127.1.124.131:32955)
0808 13:08:46.159525 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:46.159533 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:46.159631 (+ 98us) server_negotiation.cc:407] Connection header received
0808 13:08:46.159856 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:46.159861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:46.159934 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:46.160055 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:46.161489 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:46.162253 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:46.163711 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:46.164180 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:46.164316 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:46.169210 (+ 4894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:46.169226 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:46.169243 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:46.169280 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:46.172332 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:46.173110 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:46.173121 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:46.173131 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:46.173315 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:46.173715 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:46.173721 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:46.173724 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:46.174082 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:46.174407 (+ 325us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:46.174721 (+ 314us) server_negotiation.cc:299] Negotiation successful
0808 13:08:46.174944 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 13:08:46.176012 32524 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:46.267308 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 32343
I20260808 13:08:46.291636 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 31811
I20260808 13:08:46.292938 31896 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:46.407264 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 31811
2026-08-08T13:08:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4842 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T13:08:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:08:46Z Disabled control of system clock
I20260808 13:08:46.513341 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:38415
--webserver_interface=127.1.124.190
--webserver_port=0
--builtin_ntp_servers=127.1.124.148:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:38415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 13:08:46.873183 32609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:46.873504 32609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:46.873598 32609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:46.881080 32609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:08:46.881215 32609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:46.881253 32609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:46.881306 32609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:08:46.881354 32609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:08:46.890900 32609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:38415
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:38415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:46.892257 32609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:46.894240 32609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:46.906881 32626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:46.906944 32621 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:46.907097 32619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:46.907630 32609 server_base.cc:1034] running on GCE node
I20260808 13:08:46.908669 32609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:46.910190 32609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:46.911450 32609 hybrid_clock.cc:648] HybridClock initialized: now 1786194526911400 us; error 61 us; skew 500 ppm
I20260808 13:08:46.911842 32609 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:08:46.914367 32609 webserver.cc:460] Webserver started at http://127.1.124.190:37395/ using document root <none> and password file <none>
I20260808 13:08:46.915110 32609 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:46.915261 32609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:46.915604 32609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:46.918735 32609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "51a57ec7b02b4d66bf0464bf65ca0e63"
format_stamp: "Formatted at 2026-08-08 13:08:46 on dist-test-slave-t3q3"
server_key: "9d6063d3543de5403e10e867263a6a87"
server_key_iv: "5a80742c7e3adf10bdc15d28a53c602b"
server_key_version: "encryptionkey@0"
I20260808 13:08:46.919613 32609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "51a57ec7b02b4d66bf0464bf65ca0e63"
format_stamp: "Formatted at 2026-08-08 13:08:46 on dist-test-slave-t3q3"
server_key: "9d6063d3543de5403e10e867263a6a87"
server_key_iv: "5a80742c7e3adf10bdc15d28a53c602b"
server_key_version: "encryptionkey@0"
I20260808 13:08:46.925015 32609 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260808 13:08:46.928718 32634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:46.930255 32609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:08:46.930558 32609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "51a57ec7b02b4d66bf0464bf65ca0e63"
format_stamp: "Formatted at 2026-08-08 13:08:46 on dist-test-slave-t3q3"
server_key: "9d6063d3543de5403e10e867263a6a87"
server_key_iv: "5a80742c7e3adf10bdc15d28a53c602b"
server_key_version: "encryptionkey@0"
I20260808 13:08:46.930805 32609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:46.956930 32609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:46.957887 32609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:46.958222 32609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:46.978189 32609 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:38415
I20260808 13:08:46.978230 32703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:38415 every 8 connection(s)
I20260808 13:08:46.979653 32609 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:08:46.983726 32704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:46.986331 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 32609
I20260808 13:08:46.986626 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:08:46.987092 1522 external_mini_cluster.cc:1428] Setting key b74a49f97e17cf6a143ac24d0c1040ad
I20260808 13:08:46.996232 32704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63: Bootstrap starting.
I20260808 13:08:47.000180 32704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:47.001466 32704 log.cc:824] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:47.002035 32711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:46.989171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52798 (local address 127.1.124.190:38415)
0808 13:08:46.990193 (+ 1022us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:46.990205 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:46.990247 (+ 42us) server_negotiation.cc:407] Connection header received
0808 13:08:46.992040 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:46.992114 (+ 74us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:46.992644 (+ 530us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:46.992996 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:46.993661 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:46.995018 (+ 1357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:46.997031 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:46.997762 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:46.997934 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:46.999345 (+ 1411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:46.999353 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:46.999368 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:46.999945 (+ 577us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:47.000045 (+ 100us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:47.000175 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:47.000274 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:47.000532 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 13:08:47.000909 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":404,"threads_started":1}
I20260808 13:08:47.004720 32704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63: No bootstrap required, opened a new log
I20260808 13:08:47.012524 32704 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a57ec7b02b4d66bf0464bf65ca0e63" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38415 } }
I20260808 13:08:47.012888 32704 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:47.012984 32704 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51a57ec7b02b4d66bf0464bf65ca0e63, State: Initialized, Role: FOLLOWER
I20260808 13:08:47.013479 32704 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a57ec7b02b4d66bf0464bf65ca0e63" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38415 } }
I20260808 13:08:47.013681 32704 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:08:47.013782 32704 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:08:47.013896 32704 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:47.016134 32704 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a57ec7b02b4d66bf0464bf65ca0e63" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38415 } }
I20260808 13:08:47.016790 32704 leader_election.cc:302] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51a57ec7b02b4d66bf0464bf65ca0e63; no voters:
I20260808 13:08:47.017390 32704 leader_election.cc:288] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:08:47.017572 32713 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:08:47.018396 32713 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63 [term 1 LEADER]: Becoming Leader. State: Replica: 51a57ec7b02b4d66bf0464bf65ca0e63, State: Running, Role: LEADER
I20260808 13:08:47.018980 32713 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a57ec7b02b4d66bf0464bf65ca0e63" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38415 } }
I20260808 13:08:47.019498 32704 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:08:47.022173 32716 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51a57ec7b02b4d66bf0464bf65ca0e63. Latest consensus state: current_term: 1 leader_uuid: "51a57ec7b02b4d66bf0464bf65ca0e63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a57ec7b02b4d66bf0464bf65ca0e63" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38415 } } }
I20260808 13:08:47.022265 32714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51a57ec7b02b4d66bf0464bf65ca0e63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51a57ec7b02b4d66bf0464bf65ca0e63" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38415 } } }
I20260808 13:08:47.022583 32716 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63 [sys.catalog]: This master's current role is: LEADER
I20260808 13:08:47.022686 32714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63 [sys.catalog]: This master's current role is: LEADER
I20260808 13:08:47.029374 32725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:08:47.035075 32725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:08:47.045172 32725 catalog_manager.cc:1347] Generated new cluster ID: 6673c468445c4da5842aeda719324f19
I20260808 13:08:47.045307 32725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:08:47.067848 32725 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:08:47.069322 32725 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:08:47.076453 32725 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63: Generated new TSK 0
I20260808 13:08:47.077349 32725 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:08:47.092386 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.129:0
--local_ip_for_outbound_sockets=127.1.124.129
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:38415
--builtin_ntp_servers=127.1.124.148:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 13:08:47.452654 32743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:47.453050 32743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:47.453225 32743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:47.460788 32743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:47.460932 32743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:47.461124 32743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.129
I20260808 13:08:47.470547 32743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.1.124.129
--webserver_port=0
--tserver_master_addrs=127.1.124.190:38415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.129
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:47.471994 32743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:47.473942 32743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:47.487854 32754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:47.487881 32751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:47.487929 32748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:47.488698 32743 server_base.cc:1034] running on GCE node
I20260808 13:08:47.489382 32743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:47.490396 32743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:47.491649 32743 hybrid_clock.cc:648] HybridClock initialized: now 1786194527491590 us; error 53 us; skew 500 ppm
I20260808 13:08:47.491989 32743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:08:47.494506 32743 webserver.cc:460] Webserver started at http://127.1.124.129:35537/ using document root <none> and password file <none>
I20260808 13:08:47.495237 32743 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:47.495375 32743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:47.495720 32743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:47.498785 32743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-0/data/instance:
uuid: "5592972ccb804dc0919ee49c2cc83ca5"
format_stamp: "Formatted at 2026-08-08 13:08:47 on dist-test-slave-t3q3"
server_key: "b0b91640188fc0f68388d4b52fdbe6cd"
server_key_iv: "67fbf8e63a832ccf619f98253ea7ae79"
server_key_version: "encryptionkey@0"
I20260808 13:08:47.499615 32743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance:
uuid: "5592972ccb804dc0919ee49c2cc83ca5"
format_stamp: "Formatted at 2026-08-08 13:08:47 on dist-test-slave-t3q3"
server_key: "b0b91640188fc0f68388d4b52fdbe6cd"
server_key_iv: "67fbf8e63a832ccf619f98253ea7ae79"
server_key_version: "encryptionkey@0"
I20260808 13:08:47.505899 32743 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260808 13:08:47.509812 32763 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:47.511246 32743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:08:47.511449 32743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-0/wal
uuid: "5592972ccb804dc0919ee49c2cc83ca5"
format_stamp: "Formatted at 2026-08-08 13:08:47 on dist-test-slave-t3q3"
server_key: "b0b91640188fc0f68388d4b52fdbe6cd"
server_key_iv: "67fbf8e63a832ccf619f98253ea7ae79"
server_key_version: "encryptionkey@0"
I20260808 13:08:47.511695 32743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:47.558847 32743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:47.559820 32743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:47.560222 32743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:47.561345 32743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:08:47.562911 32743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:08:47.563040 32743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:47.563208 32743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:08:47.563318 32743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:47.598425 32743 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.129:44671
I20260808 13:08:47.598476 465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.129:44671 every 8 connection(s)
I20260808 13:08:47.599833 32743 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-0/data/info.pb
I20260808 13:08:47.609292 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 32743
I20260808 13:08:47.609478 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-0/wal/instance
I20260808 13:08:47.610118 1522 external_mini_cluster.cc:1428] Setting key 9a933c6a32a5eadca9a2fe9f05f1cce7
I20260808 13:08:47.613297 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.130:0
--local_ip_for_outbound_sockets=127.1.124.130
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:38415
--builtin_ntp_servers=127.1.124.148:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 13:08:47.613765 468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:47.602623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.129:52703 (local address 127.1.124.190:38415)
0808 13:08:47.603083 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:47.603088 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:47.604378 (+ 1290us) server_negotiation.cc:407] Connection header received
0808 13:08:47.606269 (+ 1891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:47.606275 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:47.606363 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:47.606502 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:47.607602 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:47.608410 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:47.610811 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:47.611273 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:47.611410 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:47.612857 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:47.612861 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:47.612865 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:47.612881 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:47.612923 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:47.612934 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:47.613015 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:47.613302 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 13:08:47.613383 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":227,"threads_started":1}
I20260808 13:08:47.615162 469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:47.603238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38415 (local address 127.1.124.129:52703)
0808 13:08:47.604199 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:47.604238 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:47.605977 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:47.606752 (+ 775us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:47.606761 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:47.606848 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:08:47.607417 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:47.607434 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:47.608583 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:47.608587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:47.610647 (+ 2060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:47.610664 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:47.611514 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:47.611522 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:47.611761 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:47.612481 (+ 720us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:08:47.612485 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:08:47.612537 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:08:47.612644 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:08:47.613054 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:08:47.613069 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:08:47.613078 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 13:08:47.613278 (+ 200us) client_negotiation.cc:240] Negotiation successful
0808 13:08:47.613715 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":289,"threads_started":1}
I20260808 13:08:47.616987 466 heartbeater.cc:347] Connected to a master server at 127.1.124.190:38415
I20260808 13:08:47.617304 466 heartbeater.cc:464] Registering TS with master...
I20260808 13:08:47.618039 466 heartbeater.cc:511] Master 127.1.124.190:38415 requested a full tablet report, sending...
I20260808 13:08:47.620260 32655 ts_manager.cc:194] Registered new tserver with Master: 5592972ccb804dc0919ee49c2cc83ca5 (127.1.124.129:44671)
I20260808 13:08:47.621793 32655 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.124.129:52703
W20260808 13:08:47.979691 470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:47.979974 470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:47.980152 470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:47.987650 470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:47.987769 470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:47.987955 470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.130
I20260808 13:08:47.997506 470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.1.124.130
--webserver_port=0
--tserver_master_addrs=127.1.124.190:38415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.130
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:47.998848 470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:48.000799 470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:48.014226 475 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:48.014284 476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:48.014276 478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:48.015206 470 server_base.cc:1034] running on GCE node
I20260808 13:08:48.015823 470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:48.016986 470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:48.018240 470 hybrid_clock.cc:648] HybridClock initialized: now 1786194528018162 us; error 51 us; skew 500 ppm
I20260808 13:08:48.018569 470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:08:48.021040 470 webserver.cc:460] Webserver started at http://127.1.124.130:42509/ using document root <none> and password file <none>
I20260808 13:08:48.021751 470 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:48.021888 470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:48.022229 470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:48.025136 470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-1/data/instance:
uuid: "2cdfe9b1666a43a6b72b1f14ed4a398e"
format_stamp: "Formatted at 2026-08-08 13:08:48 on dist-test-slave-t3q3"
server_key: "91b4879336e5140ab04c73bd68a35262"
server_key_iv: "adcdc299f2cafbb70fa074007ac2bac8"
server_key_version: "encryptionkey@0"
I20260808 13:08:48.025947 470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance:
uuid: "2cdfe9b1666a43a6b72b1f14ed4a398e"
format_stamp: "Formatted at 2026-08-08 13:08:48 on dist-test-slave-t3q3"
server_key: "91b4879336e5140ab04c73bd68a35262"
server_key_iv: "adcdc299f2cafbb70fa074007ac2bac8"
server_key_version: "encryptionkey@0"
I20260808 13:08:48.031123 470 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 13:08:48.034826 491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:48.036340 470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 13:08:48.036512 470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-1/wal
uuid: "2cdfe9b1666a43a6b72b1f14ed4a398e"
format_stamp: "Formatted at 2026-08-08 13:08:48 on dist-test-slave-t3q3"
server_key: "91b4879336e5140ab04c73bd68a35262"
server_key_iv: "adcdc299f2cafbb70fa074007ac2bac8"
server_key_version: "encryptionkey@0"
I20260808 13:08:48.036728 470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:48.060518 470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:48.061388 470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:48.061717 470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:48.062696 470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:08:48.064194 470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:08:48.064328 470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:48.064465 470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:08:48.064560 470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:48.099706 470 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.130:33985
I20260808 13:08:48.099819 657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.130:33985 every 8 connection(s)
I20260808 13:08:48.101269 470 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-1/data/info.pb
I20260808 13:08:48.111305 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 470
I20260808 13:08:48.111511 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-1/wal/instance
I20260808 13:08:48.112142 1522 external_mini_cluster.cc:1428] Setting key bb9eadb91ccf3e209a66599742897848
I20260808 13:08:48.114984 468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:48.104430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.130:40685 (local address 127.1.124.190:38415)
0808 13:08:48.104653 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:48.104661 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:48.106079 (+ 1418us) server_negotiation.cc:407] Connection header received
0808 13:08:48.107913 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:48.107918 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:48.107989 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:48.108171 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:48.109253 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.110025 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.111781 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.112428 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.112567 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:48.114090 (+ 1523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:48.114099 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:48.114104 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:48.114130 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:48.114162 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:48.114175 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:48.114244 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:48.114539 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 13:08:48.114659 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 13:08:48.115624 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.1.124.131:0
--local_ip_for_outbound_sockets=127.1.124.131
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:38415
--builtin_ntp_servers=127.1.124.148:39527
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 13:08:48.116237 661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:48.105006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38415 (local address 127.1.124.130:40685)
0808 13:08:48.105921 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:48.105958 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:48.107628 (+ 1670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:48.108368 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:48.108376 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:48.108473 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:08:48.109089 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:48.109107 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.110169 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.110173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:48.111617 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:48.111634 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.112643 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.112648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:48.112954 (+ 306us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:48.113789 (+ 835us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:08:48.113792 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:08:48.113834 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:08:48.113923 (+ 89us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:08:48.114279 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:08:48.114292 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:08:48.114300 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 13:08:48.114490 (+ 190us) client_negotiation.cc:240] Negotiation successful
0808 13:08:48.114821 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":259,"threads_started":1}
I20260808 13:08:48.117841 659 heartbeater.cc:347] Connected to a master server at 127.1.124.190:38415
I20260808 13:08:48.118167 659 heartbeater.cc:464] Registering TS with master...
I20260808 13:08:48.118932 659 heartbeater.cc:511] Master 127.1.124.190:38415 requested a full tablet report, sending...
I20260808 13:08:48.120702 32655 ts_manager.cc:194] Registered new tserver with Master: 2cdfe9b1666a43a6b72b1f14ed4a398e (127.1.124.130:33985)
I20260808 13:08:48.121917 32655 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.124.130:40685
W20260808 13:08:48.469403 662 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:48.469691 662 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:48.469842 662 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:48.477774 662 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:48.477881 662 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:48.478070 662 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.1.124.131
I20260808 13:08:48.487210 662 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:39527
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.1.124.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.1.124.131
--webserver_port=0
--tserver_master_addrs=127.1.124.190:38415
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.1.124.131
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:48.488689 662 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:48.490455 662 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:48.503636 671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:48.504272 668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:48.504217 674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:48.505360 662 server_base.cc:1034] running on GCE node
I20260808 13:08:48.506057 662 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:48.507048 662 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:48.508294 662 hybrid_clock.cc:648] HybridClock initialized: now 1786194528508236 us; error 48 us; skew 500 ppm
I20260808 13:08:48.508661 662 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260808 13:08:48.511089 662 webserver.cc:460] Webserver started at http://127.1.124.131:45635/ using document root <none> and password file <none>
I20260808 13:08:48.511924 662 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:48.512117 662 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:48.512465 662 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:48.515508 662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-2/data/instance:
uuid: "1f86783108cc49b7a37f0f325c926f2d"
format_stamp: "Formatted at 2026-08-08 13:08:48 on dist-test-slave-t3q3"
server_key: "dca1eefee9630c77436adc36fd05f4d5"
server_key_iv: "608ef9799e5363d8aaa8f80487ceebf7"
server_key_version: "encryptionkey@0"
I20260808 13:08:48.516458 662 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance:
uuid: "1f86783108cc49b7a37f0f325c926f2d"
format_stamp: "Formatted at 2026-08-08 13:08:48 on dist-test-slave-t3q3"
server_key: "dca1eefee9630c77436adc36fd05f4d5"
server_key_iv: "608ef9799e5363d8aaa8f80487ceebf7"
server_key_version: "encryptionkey@0"
I20260808 13:08:48.521802 662 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:08:48.525254 681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:48.526477 662 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260808 13:08:48.526669 662 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-2/wal
uuid: "1f86783108cc49b7a37f0f325c926f2d"
format_stamp: "Formatted at 2026-08-08 13:08:48 on dist-test-slave-t3q3"
server_key: "dca1eefee9630c77436adc36fd05f4d5"
server_key_iv: "608ef9799e5363d8aaa8f80487ceebf7"
server_key_version: "encryptionkey@0"
I20260808 13:08:48.526903 662 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:48.542302 662 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:48.543162 662 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:48.543494 662 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:48.544589 662 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 13:08:48.546203 662 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 13:08:48.546346 662 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:48.546494 662 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 13:08:48.546595 662 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:48.581122 662 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.131:44283
I20260808 13:08:48.581216 848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.131:44283 every 8 connection(s)
I20260808 13:08:48.582489 662 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-2/data/info.pb
I20260808 13:08:48.590659 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 662
I20260808 13:08:48.590832 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-2/wal/instance
I20260808 13:08:48.591321 1522 external_mini_cluster.cc:1428] Setting key f68bc4d4c349265d6940f61cd72fdeff
I20260808 13:08:48.596169 468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:48.585312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:33153 (local address 127.1.124.190:38415)
0808 13:08:48.585540 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:48.585547 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:48.586990 (+ 1443us) server_negotiation.cc:407] Connection header received
0808 13:08:48.588852 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:48.588857 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:48.588948 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:48.589077 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:48.590446 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.591321 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.593348 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.593844 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.593969 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:48.595290 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:48.595303 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:48.595307 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:48.595324 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:48.595355 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:48.595367 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:48.595464 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:48.595700 (+ 236us) server_negotiation.cc:299] Negotiation successful
0808 13:08:48.595795 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260808 13:08:48.597110 854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:48.585883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:38415 (local address 127.1.124.131:33153)
0808 13:08:48.586813 (+ 930us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:48.586852 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:48.588596 (+ 1744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:48.589345 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:48.589353 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:48.589489 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:08:48.590261 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:48.590285 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.591509 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.591514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:48.593196 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:48.593211 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.594050 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.594057 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:48.594309 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:48.594945 (+ 636us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 13:08:48.594948 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:08:48.594987 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 13:08:48.595059 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:08:48.595464 (+ 405us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:08:48.595477 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:08:48.595487 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 13:08:48.595686 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 13:08:48.596060 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":287,"threads_started":1}
I20260808 13:08:48.598527 852 heartbeater.cc:347] Connected to a master server at 127.1.124.190:38415
I20260808 13:08:48.598819 852 heartbeater.cc:464] Registering TS with master...
I20260808 13:08:48.599553 852 heartbeater.cc:511] Master 127.1.124.190:38415 requested a full tablet report, sending...
I20260808 13:08:48.601019 32655 ts_manager.cc:194] Registered new tserver with Master: 1f86783108cc49b7a37f0f325c926f2d (127.1.124.131:44283)
I20260808 13:08:48.601827 32655 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.1.124.131:33153
I20260808 13:08:48.606169 1522 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 13:08:48.616469 468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:48.610263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52808 (local address 127.1.124.190:38415)
0808 13:08:48.610491 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:48.610500 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:48.610830 (+ 330us) server_negotiation.cc:407] Connection header received
0808 13:08:48.611030 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:48.611036 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:48.611114 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:48.611224 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:48.611706 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.612594 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.614053 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.614465 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.614623 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:48.615670 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:48.615683 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:48.615687 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:48.615704 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:48.615735 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:48.615747 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:48.615873 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:48.616043 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 13:08:48.616164 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 13:08:48.624640 466 heartbeater.cc:503] Master 127.1.124.190:38415 was elected leader, sending a full tablet report...
I20260808 13:08:48.620270 32655 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:52808:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 13:08:48.627286 32655 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 13:08:48.651500 869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:48.640833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:33985 (local address 127.0.0.1:37560)
0808 13:08:48.641686 (+ 853us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:48.641758 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:48.641967 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:48.643047 (+ 1080us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:48.643055 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:48.643087 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:48.643472 (+ 385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:48.643486 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.645871 (+ 2385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.645877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:48.648629 (+ 2752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:48.648640 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.650264 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.650271 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:48.650495 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:48.650560 (+ 65us) client_negotiation.cc:769] Sending connection context
0808 13:08:48.650672 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 13:08:48.650978 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":257,"thread_start_us":303,"threads_started":1}
I20260808 13:08:48.651680 870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:48.641140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37560 (local address 127.1.124.130:33985)
0808 13:08:48.641715 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:48.641725 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:48.641864 (+ 139us) server_negotiation.cc:407] Connection header received
0808 13:08:48.642243 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:48.642251 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:48.642512 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:48.642745 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:48.643670 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.645698 (+ 2028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.648819 (+ 3121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.650024 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.650818 (+ 794us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:48.651039 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:48.651123 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 13:08:48.651284 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":280,"threads_started":1}
I20260808 13:08:48.652551 872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:48.642461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46700 (local address 127.1.124.129:44671)
0808 13:08:48.643643 (+ 1182us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:48.643662 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:48.643691 (+ 29us) server_negotiation.cc:407] Connection header received
0808 13:08:48.643816 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:48.643825 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:48.644083 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:48.644827 (+ 744us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:48.645008 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.646094 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.648934 (+ 2840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.650382 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.650521 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:48.650844 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:48.651618 (+ 774us) server_negotiation.cc:299] Negotiation successful
0808 13:08:48.651788 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":987,"thread_start_us":817,"threads_started":1}
I20260808 13:08:48.652866 871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:48.642003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:44671 (local address 127.0.0.1:46700)
0808 13:08:48.642935 (+ 932us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:48.642959 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:48.643074 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:48.644309 (+ 1235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:48.644313 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:48.644346 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:48.644664 (+ 318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:48.644672 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.646251 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.646259 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:48.648641 (+ 2382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:48.648660 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.650977 (+ 2317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.650983 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:48.651207 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:48.651246 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 13:08:48.651345 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 13:08:48.651494 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":811,"mutex_wait_us":83,"thread_start_us":649,"threads_started":1}
I20260808 13:08:48.655422 876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:48.643837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.131:44283 (local address 127.0.0.1:44988)
0808 13:08:48.646700 (+ 2863us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:48.646728 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:48.646854 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:48.647756 (+ 902us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:48.647761 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:48.647784 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:48.648247 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:48.648260 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.650271 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.650277 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:48.652954 (+ 2677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:48.652987 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.654649 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.654655 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:48.654831 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:48.654873 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 13:08:48.654961 (+ 88us) client_negotiation.cc:240] Negotiation successful
0808 13:08:48.655091 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2735,"mutex_wait_us":4,"thread_start_us":2562,"threads_started":1}
I20260808 13:08:48.655617 874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:48.643412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44988 (local address 127.1.124.131:44283)
0808 13:08:48.644242 (+ 830us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:48.644252 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:48.646994 (+ 2742us) server_negotiation.cc:407] Connection header received
0808 13:08:48.647109 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:48.647116 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:48.647330 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:48.647504 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:48.648437 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.650073 (+ 1636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.653230 (+ 3157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.654435 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.654613 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:48.654888 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:48.655013 (+ 125us) server_negotiation.cc:299] Negotiation successful
0808 13:08:48.655188 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"spinlock_wait_cycles":20224,"thread_start_us":544,"threads_started":1}
I20260808 13:08:48.656045 561 tablet_service.cc:1467] Processing CreateTablet for tablet 493777e5f11943b1a5f7b7b94e8ca727 (DEFAULT_TABLE table=test-table [id=d890bda1ca504da28b35d6a1f9a5a6a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:08:48.656535 369 tablet_service.cc:1467] Processing CreateTablet for tablet 493777e5f11943b1a5f7b7b94e8ca727 (DEFAULT_TABLE table=test-table [id=d890bda1ca504da28b35d6a1f9a5a6a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:08:48.657640 561 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 493777e5f11943b1a5f7b7b94e8ca727. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:48.657930 369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 493777e5f11943b1a5f7b7b94e8ca727. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:48.658654 751 tablet_service.cc:1467] Processing CreateTablet for tablet 493777e5f11943b1a5f7b7b94e8ca727 (DEFAULT_TABLE table=test-table [id=d890bda1ca504da28b35d6a1f9a5a6a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 13:08:48.659725 751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 493777e5f11943b1a5f7b7b94e8ca727. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:48.670941 877 tablet_bootstrap.cc:492] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d: Bootstrap starting.
I20260808 13:08:48.672916 879 tablet_bootstrap.cc:492] T 493777e5f11943b1a5f7b7b94e8ca727 P 2cdfe9b1666a43a6b72b1f14ed4a398e: Bootstrap starting.
I20260808 13:08:48.675714 878 tablet_bootstrap.cc:492] T 493777e5f11943b1a5f7b7b94e8ca727 P 5592972ccb804dc0919ee49c2cc83ca5: Bootstrap starting.
I20260808 13:08:48.676186 877 tablet_bootstrap.cc:654] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:48.677572 877 log.cc:824] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:48.679278 878 tablet_bootstrap.cc:654] T 493777e5f11943b1a5f7b7b94e8ca727 P 5592972ccb804dc0919ee49c2cc83ca5: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:48.679461 879 tablet_bootstrap.cc:654] T 493777e5f11943b1a5f7b7b94e8ca727 P 2cdfe9b1666a43a6b72b1f14ed4a398e: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:48.680608 878 log.cc:824] T 493777e5f11943b1a5f7b7b94e8ca727 P 5592972ccb804dc0919ee49c2cc83ca5: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:48.680718 879 log.cc:824] T 493777e5f11943b1a5f7b7b94e8ca727 P 2cdfe9b1666a43a6b72b1f14ed4a398e: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:48.681313 877 tablet_bootstrap.cc:492] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d: No bootstrap required, opened a new log
I20260808 13:08:48.681734 877 ts_tablet_manager.cc:1397] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260808 13:08:48.683492 879 tablet_bootstrap.cc:492] T 493777e5f11943b1a5f7b7b94e8ca727 P 2cdfe9b1666a43a6b72b1f14ed4a398e: No bootstrap required, opened a new log
I20260808 13:08:48.683763 879 ts_tablet_manager.cc:1397] T 493777e5f11943b1a5f7b7b94e8ca727 P 2cdfe9b1666a43a6b72b1f14ed4a398e: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260808 13:08:48.686205 878 tablet_bootstrap.cc:492] T 493777e5f11943b1a5f7b7b94e8ca727 P 5592972ccb804dc0919ee49c2cc83ca5: No bootstrap required, opened a new log
I20260808 13:08:48.686487 878 ts_tablet_manager.cc:1397] T 493777e5f11943b1a5f7b7b94e8ca727 P 5592972ccb804dc0919ee49c2cc83ca5: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.000s
I20260808 13:08:48.691483 879 raft_consensus.cc:348] T 493777e5f11943b1a5f7b7b94e8ca727 P 2cdfe9b1666a43a6b72b1f14ed4a398e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cdfe9b1666a43a6b72b1f14ed4a398e" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33985 } } peers { permanent_uuid: "5592972ccb804dc0919ee49c2cc83ca5" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 44671 } } peers { permanent_uuid: "1f86783108cc49b7a37f0f325c926f2d" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44283 } }
I20260808 13:08:48.691890 879 raft_consensus.cc:374] T 493777e5f11943b1a5f7b7b94e8ca727 P 2cdfe9b1666a43a6b72b1f14ed4a398e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:48.691999 879 raft_consensus.cc:729] T 493777e5f11943b1a5f7b7b94e8ca727 P 2cdfe9b1666a43a6b72b1f14ed4a398e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cdfe9b1666a43a6b72b1f14ed4a398e, State: Initialized, Role: FOLLOWER
I20260808 13:08:48.692204 877 raft_consensus.cc:348] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cdfe9b1666a43a6b72b1f14ed4a398e" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33985 } } peers { permanent_uuid: "5592972ccb804dc0919ee49c2cc83ca5" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 44671 } } peers { permanent_uuid: "1f86783108cc49b7a37f0f325c926f2d" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44283 } }
I20260808 13:08:48.692667 877 raft_consensus.cc:374] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:48.692765 877 raft_consensus.cc:729] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f86783108cc49b7a37f0f325c926f2d, State: Initialized, Role: FOLLOWER
I20260808 13:08:48.692781 879 consensus_queue.cc:260] T 493777e5f11943b1a5f7b7b94e8ca727 P 2cdfe9b1666a43a6b72b1f14ed4a398e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cdfe9b1666a43a6b72b1f14ed4a398e" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33985 } } peers { permanent_uuid: "5592972ccb804dc0919ee49c2cc83ca5" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 44671 } } peers { permanent_uuid: "1f86783108cc49b7a37f0f325c926f2d" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44283 } }
I20260808 13:08:48.693377 877 consensus_queue.cc:260] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cdfe9b1666a43a6b72b1f14ed4a398e" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33985 } } peers { permanent_uuid: "5592972ccb804dc0919ee49c2cc83ca5" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 44671 } } peers { permanent_uuid: "1f86783108cc49b7a37f0f325c926f2d" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44283 } }
I20260808 13:08:48.694126 659 heartbeater.cc:503] Master 127.1.124.190:38415 was elected leader, sending a full tablet report...
I20260808 13:08:48.694829 879 ts_tablet_manager.cc:1428] T 493777e5f11943b1a5f7b7b94e8ca727 P 2cdfe9b1666a43a6b72b1f14ed4a398e: Time spent starting tablet: real 0.011s user 0.007s sys 0.004s
I20260808 13:08:48.695858 852 heartbeater.cc:503] Master 127.1.124.190:38415 was elected leader, sending a full tablet report...
I20260808 13:08:48.696573 877 ts_tablet_manager.cc:1428] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d: Time spent starting tablet: real 0.015s user 0.009s sys 0.004s
I20260808 13:08:48.697877 878 raft_consensus.cc:348] T 493777e5f11943b1a5f7b7b94e8ca727 P 5592972ccb804dc0919ee49c2cc83ca5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cdfe9b1666a43a6b72b1f14ed4a398e" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33985 } } peers { permanent_uuid: "5592972ccb804dc0919ee49c2cc83ca5" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 44671 } } peers { permanent_uuid: "1f86783108cc49b7a37f0f325c926f2d" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44283 } }
I20260808 13:08:48.698421 878 raft_consensus.cc:374] T 493777e5f11943b1a5f7b7b94e8ca727 P 5592972ccb804dc0919ee49c2cc83ca5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:48.698571 878 raft_consensus.cc:729] T 493777e5f11943b1a5f7b7b94e8ca727 P 5592972ccb804dc0919ee49c2cc83ca5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5592972ccb804dc0919ee49c2cc83ca5, State: Initialized, Role: FOLLOWER
I20260808 13:08:48.699318 878 consensus_queue.cc:260] T 493777e5f11943b1a5f7b7b94e8ca727 P 5592972ccb804dc0919ee49c2cc83ca5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cdfe9b1666a43a6b72b1f14ed4a398e" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33985 } } peers { permanent_uuid: "5592972ccb804dc0919ee49c2cc83ca5" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 44671 } } peers { permanent_uuid: "1f86783108cc49b7a37f0f325c926f2d" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44283 } }
I20260808 13:08:48.701502 878 ts_tablet_manager.cc:1428] T 493777e5f11943b1a5f7b7b94e8ca727 P 5592972ccb804dc0919ee49c2cc83ca5: Time spent starting tablet: real 0.015s user 0.016s sys 0.000s
I20260808 13:08:48.713725 885 raft_consensus.cc:482] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 13:08:48.714128 885 raft_consensus.cc:504] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cdfe9b1666a43a6b72b1f14ed4a398e" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33985 } } peers { permanent_uuid: "5592972ccb804dc0919ee49c2cc83ca5" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 44671 } } peers { permanent_uuid: "1f86783108cc49b7a37f0f325c926f2d" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44283 } }
I20260808 13:08:48.715924 885 leader_election.cc:288] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2cdfe9b1666a43a6b72b1f14ed4a398e (127.1.124.130:33985), 5592972ccb804dc0919ee49c2cc83ca5 (127.1.124.129:44671)
I20260808 13:08:48.723136 870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:48.716520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:38851 (local address 127.1.124.130:33985)
0808 13:08:48.716771 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:48.716780 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:48.716802 (+ 22us) server_negotiation.cc:407] Connection header received
0808 13:08:48.716881 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:48.716886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:48.716983 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:48.717096 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:48.717802 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.718745 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.721188 (+ 2443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.722146 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.722315 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:48.722388 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:48.722716 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 13:08:48.722850 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 13:08:48.723177 854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:48.716354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.130:33985 (local address 127.1.124.131:38851)
0808 13:08:48.716541 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:48.716563 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:48.716712 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:48.717325 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:48.717330 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:48.717357 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:48.717607 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:48.717620 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.718886 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.718890 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:48.721025 (+ 2135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:48.721041 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.722309 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.722315 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:48.722521 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:48.722556 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 13:08:48.722660 (+ 104us) client_negotiation.cc:240] Negotiation successful
0808 13:08:48.722815 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":40}
I20260808 13:08:48.723964 872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:48.716890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.1.124.131:34591 (local address 127.1.124.129:44671)
0808 13:08:48.717166 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:48.717172 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:48.717620 (+ 448us) server_negotiation.cc:407] Connection header received
0808 13:08:48.717767 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:48.717772 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:48.717872 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:48.717979 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 13:08:48.718481 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.719269 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.721659 (+ 2390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.722821 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.722977 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:48.723046 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:48.723592 (+ 546us) server_negotiation.cc:299] Negotiation successful
0808 13:08:48.723734 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 13:08:48.724526 887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:48.716774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:44671 (local address 127.1.124.131:34591)
0808 13:08:48.717472 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:48.717492 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 13:08:48.717620 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:08:48.717996 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:08:48.717999 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:08:48.718018 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 13:08:48.718329 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:48.718338 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.719416 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.719421 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:48.721459 (+ 2038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:08:48.721470 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.723150 (+ 1680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.723159 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:08:48.723351 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:48.723384 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 13:08:48.724182 (+ 798us) client_negotiation.cc:240] Negotiation successful
0808 13:08:48.724292 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":48,"thread_start_us":341,"threads_started":1}
I20260808 13:08:48.724388 590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "493777e5f11943b1a5f7b7b94e8ca727" candidate_uuid: "1f86783108cc49b7a37f0f325c926f2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cdfe9b1666a43a6b72b1f14ed4a398e" is_pre_election: true
I20260808 13:08:48.724933 590 raft_consensus.cc:2454] T 493777e5f11943b1a5f7b7b94e8ca727 P 2cdfe9b1666a43a6b72b1f14ed4a398e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f86783108cc49b7a37f0f325c926f2d in term 0.
I20260808 13:08:48.725625 687 leader_election.cc:302] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f86783108cc49b7a37f0f325c926f2d, 2cdfe9b1666a43a6b72b1f14ed4a398e; no voters:
I20260808 13:08:48.725538 403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "493777e5f11943b1a5f7b7b94e8ca727" candidate_uuid: "1f86783108cc49b7a37f0f325c926f2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592972ccb804dc0919ee49c2cc83ca5" is_pre_election: true
I20260808 13:08:48.726032 885 raft_consensus.cc:2781] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 13:08:48.726046 403 raft_consensus.cc:2454] T 493777e5f11943b1a5f7b7b94e8ca727 P 5592972ccb804dc0919ee49c2cc83ca5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f86783108cc49b7a37f0f325c926f2d in term 0.
I20260808 13:08:48.726150 885 raft_consensus.cc:482] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 13:08:48.726282 885 raft_consensus.cc:3037] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:48.729069 885 raft_consensus.cc:504] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cdfe9b1666a43a6b72b1f14ed4a398e" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33985 } } peers { permanent_uuid: "5592972ccb804dc0919ee49c2cc83ca5" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 44671 } } peers { permanent_uuid: "1f86783108cc49b7a37f0f325c926f2d" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44283 } }
I20260808 13:08:48.729913 885 leader_election.cc:288] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d [CANDIDATE]: Term 1 election: Requested vote from peers 2cdfe9b1666a43a6b72b1f14ed4a398e (127.1.124.130:33985), 5592972ccb804dc0919ee49c2cc83ca5 (127.1.124.129:44671)
I20260808 13:08:48.730434 590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "493777e5f11943b1a5f7b7b94e8ca727" candidate_uuid: "1f86783108cc49b7a37f0f325c926f2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cdfe9b1666a43a6b72b1f14ed4a398e"
I20260808 13:08:48.730710 590 raft_consensus.cc:3037] T 493777e5f11943b1a5f7b7b94e8ca727 P 2cdfe9b1666a43a6b72b1f14ed4a398e [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:48.730757 403 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "493777e5f11943b1a5f7b7b94e8ca727" candidate_uuid: "1f86783108cc49b7a37f0f325c926f2d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5592972ccb804dc0919ee49c2cc83ca5"
I20260808 13:08:48.730993 403 raft_consensus.cc:3037] T 493777e5f11943b1a5f7b7b94e8ca727 P 5592972ccb804dc0919ee49c2cc83ca5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:48.734449 590 raft_consensus.cc:2454] T 493777e5f11943b1a5f7b7b94e8ca727 P 2cdfe9b1666a43a6b72b1f14ed4a398e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f86783108cc49b7a37f0f325c926f2d in term 1.
I20260808 13:08:48.734601 403 raft_consensus.cc:2454] T 493777e5f11943b1a5f7b7b94e8ca727 P 5592972ccb804dc0919ee49c2cc83ca5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f86783108cc49b7a37f0f325c926f2d in term 1.
I20260808 13:08:48.735009 687 leader_election.cc:302] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f86783108cc49b7a37f0f325c926f2d, 5592972ccb804dc0919ee49c2cc83ca5; no voters:
I20260808 13:08:48.735591 885 raft_consensus.cc:2781] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:08:48.736161 885 raft_consensus.cc:686] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d [term 1 LEADER]: Becoming Leader. State: Replica: 1f86783108cc49b7a37f0f325c926f2d, State: Running, Role: LEADER
I20260808 13:08:48.736668 885 consensus_queue.cc:237] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cdfe9b1666a43a6b72b1f14ed4a398e" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33985 } } peers { permanent_uuid: "5592972ccb804dc0919ee49c2cc83ca5" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 44671 } } peers { permanent_uuid: "1f86783108cc49b7a37f0f325c926f2d" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44283 } }
I20260808 13:08:48.741685 32651 catalog_manager.cc:5495] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f86783108cc49b7a37f0f325c926f2d (127.1.124.131). New cstate: current_term: 1 leader_uuid: "1f86783108cc49b7a37f0f325c926f2d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2cdfe9b1666a43a6b72b1f14ed4a398e" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33985 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5592972ccb804dc0919ee49c2cc83ca5" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 44671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f86783108cc49b7a37f0f325c926f2d" member_type: VOTER last_known_addr { host: "127.1.124.131" port: 44283 } health_report { overall_health: HEALTHY } } }
I20260808 13:08:48.772109 874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:48.766143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44994 (local address 127.1.124.131:44283)
0808 13:08:48.766403 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:48.766410 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:48.766433 (+ 23us) server_negotiation.cc:407] Connection header received
0808 13:08:48.766548 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:48.766552 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:48.766636 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:48.766821 (+ 185us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:08:48.767252 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.768098 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.769660 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.770136 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.770289 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:48.770846 (+ 557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:08:48.771408 (+ 562us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:08:48.771596 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:48.771706 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 13:08:48.771793 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 13:08:48.780786 590 raft_consensus.cc:1261] T 493777e5f11943b1a5f7b7b94e8ca727 P 2cdfe9b1666a43a6b72b1f14ed4a398e [term 1 FOLLOWER]: Refusing update from remote peer 1f86783108cc49b7a37f0f325c926f2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:08:48.780829 403 raft_consensus.cc:1261] T 493777e5f11943b1a5f7b7b94e8ca727 P 5592972ccb804dc0919ee49c2cc83ca5 [term 1 FOLLOWER]: Refusing update from remote peer 1f86783108cc49b7a37f0f325c926f2d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:08:48.781810 888 consensus_queue.cc:1035] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5592972ccb804dc0919ee49c2cc83ca5" member_type: VOTER last_known_addr { host: "127.1.124.129" port: 44671 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:08:48.782222 885 consensus_queue.cc:1035] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cdfe9b1666a43a6b72b1f14ed4a398e" member_type: VOTER last_known_addr { host: "127.1.124.130" port: 33985 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:08:48.798748 894 mvcc.cc:205] Tried to move back new op lower bound from 7316252789875220480 to 7316252789709647872. Current Snapshot: MvccSnapshot[applied={T|T < 7316252789875220480}]
I20260808 13:08:48.808995 32652 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:52808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:08:48.809352 32652 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:52808:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 13:08:48.813138 32652 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 51a57ec7b02b4d66bf0464bf65ca0e63: Sending DeleteTablet for 3 replicas of tablet 493777e5f11943b1a5f7b7b94e8ca727
I20260808 13:08:48.814329 561 tablet_service.cc:1514] Processing DeleteTablet for tablet 493777e5f11943b1a5f7b7b94e8ca727 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:08:48 UTC) from {username='slave'} at 127.0.0.1:37560
I20260808 13:08:48.814445 751 tablet_service.cc:1514] Processing DeleteTablet for tablet 493777e5f11943b1a5f7b7b94e8ca727 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:08:48 UTC) from {username='slave'} at 127.0.0.1:44988
I20260808 13:08:48.814528 369 tablet_service.cc:1514] Processing DeleteTablet for tablet 493777e5f11943b1a5f7b7b94e8ca727 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 13:08:48 UTC) from {username='slave'} at 127.0.0.1:46700
I20260808 13:08:48.816666 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 32743
I20260808 13:08:48.824380 909 tablet_replica.cc:330] T 493777e5f11943b1a5f7b7b94e8ca727 P 5592972ccb804dc0919ee49c2cc83ca5: stopping tablet replica
I20260808 13:08:48.824986 909 raft_consensus.cc:2229] T 493777e5f11943b1a5f7b7b94e8ca727 P 5592972ccb804dc0919ee49c2cc83ca5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:08:48.825672 909 raft_consensus.cc:2258] T 493777e5f11943b1a5f7b7b94e8ca727 P 5592972ccb804dc0919ee49c2cc83ca5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:08:48.829926 907 tablet_replica.cc:330] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d: stopping tablet replica
I20260808 13:08:48.830510 907 raft_consensus.cc:2229] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d [term 1 LEADER]: Raft consensus shutting down.
I20260808 13:08:48.831634 907 raft_consensus.cc:2258] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:08:48.832209 909 ts_tablet_manager.cc:1905] T 493777e5f11943b1a5f7b7b94e8ca727 P 5592972ccb804dc0919ee49c2cc83ca5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:08:48.833910 907 ts_tablet_manager.cc:1905] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:08:48.836647 905 tablet_replica.cc:330] T 493777e5f11943b1a5f7b7b94e8ca727 P 2cdfe9b1666a43a6b72b1f14ed4a398e: stopping tablet replica
I20260808 13:08:48.837152 905 raft_consensus.cc:2229] T 493777e5f11943b1a5f7b7b94e8ca727 P 2cdfe9b1666a43a6b72b1f14ed4a398e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 13:08:48.837657 905 raft_consensus.cc:2258] T 493777e5f11943b1a5f7b7b94e8ca727 P 2cdfe9b1666a43a6b72b1f14ed4a398e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 13:08:48.840643 872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:48.817817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46712 (local address 127.1.124.129:44671)
0808 13:08:48.818000 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:48.818008 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:48.830668 (+ 12660us) server_negotiation.cc:407] Connection header received
0808 13:08:48.831553 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:48.831560 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:48.831684 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:48.831768 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:48.832682 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.833906 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.836340 (+ 2434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.836962 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.837077 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:48.838772 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:48.838786 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:48.838804 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:48.839601 (+ 797us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:48.839667 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:48.839695 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:48.839794 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:48.840127 (+ 333us) server_negotiation.cc:299] Negotiation successful
0808 13:08:48.840253 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 13:08:48.842163 905 ts_tablet_manager.cc:1905] T 493777e5f11943b1a5f7b7b94e8ca727 P 2cdfe9b1666a43a6b72b1f14ed4a398e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 13:08:48.842199 460 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:48.846503 907 ts_tablet_manager.cc:1918] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:08:48.846747 907 log.cc:1196] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-2/wal/wals/493777e5f11943b1a5f7b7b94e8ca727
I20260808 13:08:48.847342 907 ts_tablet_manager.cc:1939] T 493777e5f11943b1a5f7b7b94e8ca727 P 1f86783108cc49b7a37f0f325c926f2d: Deleting consensus metadata
I20260808 13:08:48.849334 32637 catalog_manager.cc:4841] TS 1f86783108cc49b7a37f0f325c926f2d (127.1.124.131:44283): tablet 493777e5f11943b1a5f7b7b94e8ca727 (table test-table [id=d890bda1ca504da28b35d6a1f9a5a6a3]) successfully deleted
I20260808 13:08:48.858201 905 ts_tablet_manager.cc:1918] T 493777e5f11943b1a5f7b7b94e8ca727 P 2cdfe9b1666a43a6b72b1f14ed4a398e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 13:08:48.858421 905 log.cc:1196] T 493777e5f11943b1a5f7b7b94e8ca727 P 2cdfe9b1666a43a6b72b1f14ed4a398e: Deleting WAL directory at /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786194185613124-1522-0/minicluster-data/ts-1/wal/wals/493777e5f11943b1a5f7b7b94e8ca727
I20260808 13:08:48.859036 905 ts_tablet_manager.cc:1939] T 493777e5f11943b1a5f7b7b94e8ca727 P 2cdfe9b1666a43a6b72b1f14ed4a398e: Deleting consensus metadata
I20260808 13:08:48.861176 32637 catalog_manager.cc:4841] TS 2cdfe9b1666a43a6b72b1f14ed4a398e (127.1.124.130:33985): tablet 493777e5f11943b1a5f7b7b94e8ca727 (table test-table [id=d890bda1ca504da28b35d6a1f9a5a6a3]) successfully deleted
I20260808 13:08:48.935947 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 32743
W20260808 13:08:48.960358 32637 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.1.124.129:44671 (error 108)
I20260808 13:08:48.961220 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 470
I20260808 13:08:48.962894 876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:48.962382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:44671 (local address 127.0.0.1:46714)
0808 13:08:48.962563 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:48.962722 (+ 159us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:44671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260808 13:08:48.963195 32637 catalog_manager.cc:4584] TS 5592972ccb804dc0919ee49c2cc83ca5 (127.1.124.129:44671): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 493777e5f11943b1a5f7b7b94e8ca727: Network error: Client connection negotiation failed: client connection to 127.1.124.129:44671: connect: Connection refused (error 111)
I20260808 13:08:48.969439 870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:48.962060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37566 (local address 127.1.124.130:33985)
0808 13:08:48.962250 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:48.962256 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:48.962281 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:08:48.962467 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:48.962473 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:48.962599 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:48.962713 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:48.963220 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.964647 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.965946 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:48.966605 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:48.966738 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:48.967786 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:48.967797 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:48.967816 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:48.968633 (+ 817us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:48.968701 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:48.968726 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:48.968843 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:48.968965 (+ 122us) server_negotiation.cc:299] Negotiation successful
0808 13:08:48.969078 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 13:08:48.970323 651 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:49.017058 876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:49.016473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:44671 (local address 127.0.0.1:46720)
0808 13:08:49.016728 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:49.016849 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:44671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260808 13:08:49.018328 876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:49.017903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:44671 (local address 127.0.0.1:46724)
0808 13:08:49.018085 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:49.018177 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:44671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 13:08:49.073419 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 470
I20260808 13:08:49.079648 876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:49.078994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:44671 (local address 127.0.0.1:46736)
0808 13:08:49.079299 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:49.079407 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:44671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157}
I20260808 13:08:49.080937 876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:49.080485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:44671 (local address 127.0.0.1:46748)
0808 13:08:49.080687 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:49.080782 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:44671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 13:08:49.097695 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 662
I20260808 13:08:49.106611 874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:49.098664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44998 (local address 127.1.124.131:44283)
0808 13:08:49.098877 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:49.098883 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:49.098905 (+ 22us) server_negotiation.cc:407] Connection header received
0808 13:08:49.099042 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:49.099045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:49.099163 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:49.099259 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:49.099860 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:49.100793 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:49.102752 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:49.103272 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:49.103380 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:49.104756 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:49.104764 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:49.104781 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:49.105639 (+ 858us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:49.105689 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:49.105711 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:49.105799 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:49.106081 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 13:08:49.106183 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260808 13:08:49.107466 842 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:49.162338 876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:49.161622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:44671 (local address 127.0.0.1:46764)
0808 13:08:49.161937 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:49.162138 (+ 201us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:44671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138}
I20260808 13:08:49.163699 876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:49.163189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:44671 (local address 127.0.0.1:46770)
0808 13:08:49.163452 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:49.163553 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:44671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260808 13:08:49.234787 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 662
I20260808 13:08:49.262149 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 32609
I20260808 13:08:49.263273 32698 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:49.375579 876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:49.375021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.129:44671 (local address 127.0.0.1:55108)
0808 13:08:49.375189 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 13:08:49.375323 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.129:44671: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260808 13:08:49.375643 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 32609
2026-08-08T13:08:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2968 ms)
[----------] 6 tests from EncryptionPolicyTest (23799 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:08:49 dist-test-slave-t3q3 krb5kdc[929](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:08:49 dist-test-slave-t3q3 krb5kdc[929](info): set up 2 sockets
Aug 08 13:08:49 dist-test-slave-t3q3 krb5kdc[929](info): commencing operation
krb5kdc: starting...
W20260808 13:08:50.745764 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.315s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:08:50 dist-test-slave-t3q3 krb5kdc[929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:08:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:08:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:08:51.073822 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40427
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42633
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40427
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:08:51.424188 948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:51.424489 948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:51.424588 948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:51.431861 948 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:08:51.431982 948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:51.432031 948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:51.432153 948 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:08:51.432207 948 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:08:51.442188 948 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42633
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40427
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40427
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:51.443526 948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:51.444887 948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:51.456864 959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:51.456923 957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:51.456899 956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:51.457871 948 server_base.cc:1034] running on GCE node
I20260808 13:08:51.458652 948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:51.460119 948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:51.461339 948 hybrid_clock.cc:648] HybridClock initialized: now 1786194531461303 us; error 25 us; skew 500 ppm
I20260808 13:08:51.461647 948 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:08:51 dist-test-slave-t3q3 krb5kdc[929](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194531, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:08:51.469244 948 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:08:51.470829 948 webserver.cc:460] Webserver started at http://127.0.0.1:38201/ using document root <none> and password file <none>
I20260808 13:08:51.471487 948 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:51.471588 948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:51.471848 948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:51.474927 948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "f3bf883adcab42c2b712cc11c30c63bf"
format_stamp: "Formatted at 2026-08-08 13:08:51 on dist-test-slave-t3q3"
server_key: "490f2b26804f499fe75ffcd0e0a8c9b0"
server_key_iv: "f61dd3e73c220988a31d93e6cc20fd68"
server_key_version: "encryptionkey@0"
I20260808 13:08:51.475649 948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "f3bf883adcab42c2b712cc11c30c63bf"
format_stamp: "Formatted at 2026-08-08 13:08:51 on dist-test-slave-t3q3"
server_key: "490f2b26804f499fe75ffcd0e0a8c9b0"
server_key_iv: "f61dd3e73c220988a31d93e6cc20fd68"
server_key_version: "encryptionkey@0"
I20260808 13:08:51.481127 948 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 13:08:51.484678 971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:51.486011 948 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260808 13:08:51.486169 948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "f3bf883adcab42c2b712cc11c30c63bf"
format_stamp: "Formatted at 2026-08-08 13:08:51 on dist-test-slave-t3q3"
server_key: "490f2b26804f499fe75ffcd0e0a8c9b0"
server_key_iv: "f61dd3e73c220988a31d93e6cc20fd68"
server_key_version: "encryptionkey@0"
I20260808 13:08:51.486331 948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:51.524130 948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:51.528576 948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:51.528831 948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:51.548722 948 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40427
I20260808 13:08:51.548768 1050 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40427 every 8 connection(s)
I20260808 13:08:51.550243 948 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:08:51.554147 1051 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:51.555992 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 948
I20260808 13:08:51.556285 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:08:51.557191 1522 external_mini_cluster.cc:1428] Setting key 6325010caa6563b5cd75d6faca82e39a
I20260808 13:08:51.568393 1051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf: Bootstrap starting.
I20260808 13:08:51.571708 1051 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf: Neither blocks nor log segments found. Creating new log.
Aug 08 13:08:51 dist-test-slave-t3q3 krb5kdc[929](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 13:08:51.572858 1051 log.cc:824] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:51.576231 1051 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf: No bootstrap required, opened a new log
I20260808 13:08:51.581250 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:51.559367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47181 (local address 127.0.0.1:40427)
0808 13:08:51.560304 (+ 937us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:51.560317 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:51.560361 (+ 44us) server_negotiation.cc:407] Connection header received
0808 13:08:51.562086 (+ 1725us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:51.562113 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:51.562557 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:51.562994 (+ 437us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:51.564684 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:51.566100 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:51.567513 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:51.568302 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:51.568513 (+ 211us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:51.573042 (+ 4529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:51.573053 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:51.573074 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:51.573130 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:51.576667 (+ 3537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:51.577788 (+ 1121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:51.577801 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:51.577807 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:51.577977 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:51.578480 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:51.578495 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:51.578498 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:51.579029 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:51.579432 (+ 403us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:51.579885 (+ 453us) server_negotiation.cc:299] Negotiation successful
0808 13:08:51.580246 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":402,"threads_started":1}
I20260808 13:08:51.585611 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:08:51.585583 1051 raft_consensus.cc:348] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3bf883adcab42c2b712cc11c30c63bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40427 } }
I20260808 13:08:51.586016 1051 raft_consensus.cc:374] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:51.586145 1051 raft_consensus.cc:729] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3bf883adcab42c2b712cc11c30c63bf, State: Initialized, Role: FOLLOWER
I20260808 13:08:51.586714 1051 consensus_queue.cc:260] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3bf883adcab42c2b712cc11c30c63bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40427 } }
I20260808 13:08:51.586964 1051 raft_consensus.cc:388] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:08:51.587117 1051 raft_consensus.cc:482] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:08:51.587307 1051 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:51.589823 1051 raft_consensus.cc:504] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3bf883adcab42c2b712cc11c30c63bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40427 } }
I20260808 13:08:51.590436 1051 leader_election.cc:302] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f3bf883adcab42c2b712cc11c30c63bf; no voters:
I20260808 13:08:51.591202 1051 leader_election.cc:288] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:08:51.591331 1070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:08:51.593843 1051 sys_catalog.cc:564] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:08:51.598316 1070 raft_consensus.cc:686] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf [term 1 LEADER]: Becoming Leader. State: Replica: f3bf883adcab42c2b712cc11c30c63bf, State: Running, Role: LEADER
I20260808 13:08:51.599205 1070 consensus_queue.cc:237] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3bf883adcab42c2b712cc11c30c63bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40427 } }
I20260808 13:08:51.603039 1076 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader f3bf883adcab42c2b712cc11c30c63bf. Latest consensus state: current_term: 1 leader_uuid: "f3bf883adcab42c2b712cc11c30c63bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3bf883adcab42c2b712cc11c30c63bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40427 } } }
I20260808 13:08:51.603178 1075 sys_catalog.cc:455] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f3bf883adcab42c2b712cc11c30c63bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f3bf883adcab42c2b712cc11c30c63bf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40427 } } }
I20260808 13:08:51.603484 1076 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf [sys.catalog]: This master's current role is: LEADER
I20260808 13:08:51.603627 1075 sys_catalog.cc:458] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf [sys.catalog]: This master's current role is: LEADER
I20260808 13:08:51.605398 1090 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:08:51.611162 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:51.591216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33751 (local address 127.0.0.1:40427)
0808 13:08:51.591405 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:51.591412 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:51.592088 (+ 676us) server_negotiation.cc:407] Connection header received
0808 13:08:51.592428 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:51.592437 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:51.592541 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:51.592648 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:51.595142 (+ 2494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:51.596845 (+ 1703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:51.598966 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:51.599657 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:51.599802 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:51.603047 (+ 3245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:51.603057 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:51.603065 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:51.603112 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:51.607338 (+ 4226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:51.608489 (+ 1151us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:51.608497 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:51.608504 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:51.608698 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:51.609307 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:51.609311 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:51.609314 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:51.609790 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:51.610089 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:51.610589 (+ 500us) server_negotiation.cc:299] Negotiation successful
0808 13:08:51.610813 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 13:08:51.612126 1090 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:08:51.620803 1090 catalog_manager.cc:1347] Generated new cluster ID: b0027167e81a4f93a17762602e212a30
I20260808 13:08:51.620934 1090 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:08:51.639938 1090 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:08:51.641825 1090 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:08:51.660229 1090 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f3bf883adcab42c2b712cc11c30c63bf: Generated new TSK 0
I20260808 13:08:51.661350 1090 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:08:51.670171 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 948
I20260808 13:08:51.671657 1043 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:51.764811 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 948
W20260808 13:08:51.791157 927 connection.cc:535] client connection to 127.0.0.1:40427 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40427): Connection reset by peer (error 104)
2026-08-08T13:08:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2380 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:08:51 dist-test-slave-t3q3 krb5kdc[1103](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:08:51 dist-test-slave-t3q3 krb5kdc[1103](info): set up 2 sockets
Aug 08 13:08:51 dist-test-slave-t3q3 krb5kdc[1103](info): commencing operation
krb5kdc: starting...
W20260808 13:08:53.118141 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.308s user 0.001s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:08:53 dist-test-slave-t3q3 krb5kdc[1103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:08:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:08:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:08:53.452018 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46115
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:08:53.800976 1123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:53.801339 1123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:53.801520 1123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:53.808696 1123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:08:53.808862 1123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:53.808983 1123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:53.809096 1123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:08:53.809186 1123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:08:53.818955 1123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46115
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:53.820338 1123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:53.821718 1123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:53.833601 1130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:53.833784 1131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:53.833998 1134 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:53.834808 1123 server_base.cc:1034] running on GCE node
I20260808 13:08:53.835601 1123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:53.837056 1123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:53.838263 1123 hybrid_clock.cc:648] HybridClock initialized: now 1786194533838219 us; error 31 us; skew 500 ppm
I20260808 13:08:53.838625 1123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:08:53 dist-test-slave-t3q3 krb5kdc[1103](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194533, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:08:53.841980 1123 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:08:53.843568 1123 webserver.cc:460] Webserver started at http://127.0.0.1:41845/ using document root <none> and password file <none>
I20260808 13:08:53.844377 1123 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:53.844532 1123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:53.844851 1123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:53.847837 1123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "b48a3e9367384fc5a4b0cbc5d03ac296"
format_stamp: "Formatted at 2026-08-08 13:08:53 on dist-test-slave-t3q3"
server_key: "6a626443e623d51f6b8670b0fb56ca43"
server_key_iv: "99b019de813b997246bf32497d55a66e"
server_key_version: "encryptionkey@0"
I20260808 13:08:53.848680 1123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "b48a3e9367384fc5a4b0cbc5d03ac296"
format_stamp: "Formatted at 2026-08-08 13:08:53 on dist-test-slave-t3q3"
server_key: "6a626443e623d51f6b8670b0fb56ca43"
server_key_iv: "99b019de813b997246bf32497d55a66e"
server_key_version: "encryptionkey@0"
I20260808 13:08:53.854041 1123 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:08:53.857929 1147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:53.859403 1123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:08:53.859620 1123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "b48a3e9367384fc5a4b0cbc5d03ac296"
format_stamp: "Formatted at 2026-08-08 13:08:53 on dist-test-slave-t3q3"
server_key: "6a626443e623d51f6b8670b0fb56ca43"
server_key_iv: "99b019de813b997246bf32497d55a66e"
server_key_version: "encryptionkey@0"
I20260808 13:08:53.859848 1123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:53.877290 1123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:53.881572 1123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:53.881896 1123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:53.901551 1123 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46115
I20260808 13:08:53.901592 1224 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46115 every 8 connection(s)
I20260808 13:08:53.903033 1123 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:08:53.904640 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 1123
I20260808 13:08:53.904837 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:08:53.905349 1522 external_mini_cluster.cc:1428] Setting key 40484e69cc09ff3541ac5a9ad17ce069
I20260808 13:08:53.908053 1225 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:53.921828 1229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:53.907327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59019 (local address 127.0.0.1:46115)
0808 13:08:53.908221 (+ 894us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:53.908238 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:53.908290 (+ 52us) server_negotiation.cc:407] Connection header received
0808 13:08:53.910764 (+ 2474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:53.910794 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:53.911349 (+ 555us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:53.911826 (+ 477us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:53.912435 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:53.914612 (+ 2177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:53.916055 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:53.917175 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:53.917367 (+ 192us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:53.918664 (+ 1297us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:53.918675 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:53.918701 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:53.919524 (+ 823us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:53.919634 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:53.919677 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:53.919850 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:53.920219 (+ 369us) server_negotiation.cc:299] Negotiation successful
0808 13:08:53.920425 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":349,"threads_started":1}
I20260808 13:08:53.923045 1225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296: Bootstrap starting.
I20260808 13:08:53.926076 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:08:53.926853 1225 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:53.927943 1225 log.cc:824] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:53.931807 1225 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296: No bootstrap required, opened a new log
I20260808 13:08:53.939280 1229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:53.931551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45035 (local address 127.0.0.1:46115)
0808 13:08:53.931772 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:53.931781 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:53.932376 (+ 595us) server_negotiation.cc:407] Connection header received
0808 13:08:53.932596 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:53.932599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:53.932662 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:53.932747 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:53.933472 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:53.934265 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:53.936521 (+ 2256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:53.936987 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:53.937084 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:53.938433 (+ 1349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:53.938438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:53.938441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:53.938463 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:53.938503 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:53.938521 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:53.938615 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:53.938839 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 13:08:53.938886 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260808 13:08:53.942018 1225 raft_consensus.cc:348] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b48a3e9367384fc5a4b0cbc5d03ac296" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46115 } }
I20260808 13:08:53.942473 1225 raft_consensus.cc:374] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:53.942590 1225 raft_consensus.cc:729] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b48a3e9367384fc5a4b0cbc5d03ac296, State: Initialized, Role: FOLLOWER
I20260808 13:08:53.943222 1225 consensus_queue.cc:260] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b48a3e9367384fc5a4b0cbc5d03ac296" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46115 } }
I20260808 13:08:53.943499 1225 raft_consensus.cc:388] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:08:53.943655 1225 raft_consensus.cc:482] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:08:53.943843 1225 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:53.946203 1225 raft_consensus.cc:504] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b48a3e9367384fc5a4b0cbc5d03ac296" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46115 } }
I20260808 13:08:53.946759 1225 leader_election.cc:302] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b48a3e9367384fc5a4b0cbc5d03ac296; no voters:
I20260808 13:08:53.947482 1225 leader_election.cc:288] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:08:53.947594 1241 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:08:53.948316 1241 raft_consensus.cc:686] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296 [term 1 LEADER]: Becoming Leader. State: Replica: b48a3e9367384fc5a4b0cbc5d03ac296, State: Running, Role: LEADER
I20260808 13:08:53.948975 1241 consensus_queue.cc:237] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b48a3e9367384fc5a4b0cbc5d03ac296" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46115 } }
I20260808 13:08:53.950039 1225 sys_catalog.cc:564] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:08:53.957232 1243 sys_catalog.cc:455] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b48a3e9367384fc5a4b0cbc5d03ac296. Latest consensus state: current_term: 1 leader_uuid: "b48a3e9367384fc5a4b0cbc5d03ac296" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b48a3e9367384fc5a4b0cbc5d03ac296" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46115 } } }
I20260808 13:08:53.957979 1243 sys_catalog.cc:458] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296 [sys.catalog]: This master's current role is: LEADER
I20260808 13:08:53.957156 1242 sys_catalog.cc:455] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b48a3e9367384fc5a4b0cbc5d03ac296" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b48a3e9367384fc5a4b0cbc5d03ac296" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46115 } } }
I20260808 13:08:53.958531 1242 sys_catalog.cc:458] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296 [sys.catalog]: This master's current role is: LEADER
W20260808 13:08:53.962973 1259 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:08:53.963114 1259 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:08:53.963291 1261 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:08:53.965207 1261 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:08:53.973785 1261 catalog_manager.cc:1347] Generated new cluster ID: f8f84b02926741be97aac72f24366791
I20260808 13:08:53.973932 1261 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:08:53.994251 1261 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:08:53.996469 1261 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:08:54.007934 1261 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b48a3e9367384fc5a4b0cbc5d03ac296: Generated new TSK 0
I20260808 13:08:54.009073 1261 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:08:54.025825 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 1123
W20260808 13:08:54.028402 1219 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59019
W20260808 13:08:54.029280 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 1123: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:08:54.029486 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 1123
W20260808 13:08:54.056915 1101 connection.cc:535] client connection to 127.0.0.1:46115 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46115): Connection reset by peer (error 104)
2026-08-08T13:08:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2265 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:08:54 dist-test-slave-t3q3 krb5kdc[1268](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:08:54 dist-test-slave-t3q3 krb5kdc[1268](info): set up 2 sockets
Aug 08 13:08:54 dist-test-slave-t3q3 krb5kdc[1268](info): commencing operation
krb5kdc: starting...
W20260808 13:08:55.404611 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.328s user 0.001s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:08:55 dist-test-slave-t3q3 krb5kdc[1268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194535, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:08:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:08:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:08:55.736058 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34623
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46227
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34623
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:08:56.091550 1289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:56.091897 1289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:56.092090 1289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:56.099363 1289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:08:56.099524 1289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:56.099632 1289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:56.099752 1289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:08:56.099833 1289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:08:56.109864 1289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46227
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34623
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34623
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:56.111274 1289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:56.112748 1289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:56.125499 1298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:56.125548 1299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:56.125452 1302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:56.126360 1289 server_base.cc:1034] running on GCE node
I20260808 13:08:56.127246 1289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:56.128706 1289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:56.129941 1289 hybrid_clock.cc:648] HybridClock initialized: now 1786194536129907 us; error 46 us; skew 500 ppm
I20260808 13:08:56.130306 1289 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:08:56 dist-test-slave-t3q3 krb5kdc[1268](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194536, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:08:56.133571 1289 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:08:56.135084 1289 webserver.cc:460] Webserver started at http://127.0.0.1:35951/ using document root <none> and password file <none>
I20260808 13:08:56.135842 1289 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:56.135995 1289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:56.136351 1289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:56.139353 1289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "43fb8df12c744884a0ff3c438a5978fe"
format_stamp: "Formatted at 2026-08-08 13:08:56 on dist-test-slave-t3q3"
server_key: "ec3b1a2b9f4cc9641b19da2451d7de6b"
server_key_iv: "5d52ce45e4b17ad8445c108adf8ba26e"
server_key_version: "encryptionkey@0"
I20260808 13:08:56.140185 1289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "43fb8df12c744884a0ff3c438a5978fe"
format_stamp: "Formatted at 2026-08-08 13:08:56 on dist-test-slave-t3q3"
server_key: "ec3b1a2b9f4cc9641b19da2451d7de6b"
server_key_iv: "5d52ce45e4b17ad8445c108adf8ba26e"
server_key_version: "encryptionkey@0"
I20260808 13:08:56.145378 1289 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260808 13:08:56.148845 1312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:56.150097 1289 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:08:56.150305 1289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "43fb8df12c744884a0ff3c438a5978fe"
format_stamp: "Formatted at 2026-08-08 13:08:56 on dist-test-slave-t3q3"
server_key: "ec3b1a2b9f4cc9641b19da2451d7de6b"
server_key_iv: "5d52ce45e4b17ad8445c108adf8ba26e"
server_key_version: "encryptionkey@0"
I20260808 13:08:56.150517 1289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:56.157183 1289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:56.161855 1289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:56.162182 1289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:56.181577 1289 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34623
I20260808 13:08:56.181622 1392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34623 every 8 connection(s)
I20260808 13:08:56.183037 1289 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:08:56.185557 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 1289
I20260808 13:08:56.185755 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:08:56.186240 1522 external_mini_cluster.cc:1428] Setting key c6113001b566e34e3133f00e7bfdf441
I20260808 13:08:56.187103 1393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:56.194975 1397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:56.188235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53353 (local address 127.0.0.1:34623)
0808 13:08:56.189055 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:56.189070 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:56.189131 (+ 61us) server_negotiation.cc:407] Connection header received
0808 13:08:56.190869 (+ 1738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:56.190899 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:56.191375 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:56.191860 (+ 485us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:56.192915 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:56.192924 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:56.192942 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:56.192986 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:56.193076 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:56.193119 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:56.193351 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:56.193484 (+ 133us) server_negotiation.cc:299] Negotiation successful
0808 13:08:56.193683 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":318,"threads_started":1}
I20260808 13:08:56.199119 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:08:56.202318 1393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe: Bootstrap starting.
I20260808 13:08:56.206131 1393 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:56.206530 1397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:56.203750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53745 (local address 127.0.0.1:34623)
0808 13:08:56.203963 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:56.203971 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:56.204396 (+ 425us) server_negotiation.cc:407] Connection header received
0808 13:08:56.204615 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:56.204620 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:56.204678 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:56.204785 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:56.205813 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:56.205822 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:56.205826 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:08:56.205841 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:56.205879 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:08:56.205894 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:56.206013 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:56.206149 (+ 136us) server_negotiation.cc:299] Negotiation successful
0808 13:08:56.206208 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 13:08:56.207579 1393 log.cc:824] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe: Log is configured to *not* fsync() on all Append() calls
I20260808 13:08:56.210798 1393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe: No bootstrap required, opened a new log
I20260808 13:08:56.218575 1393 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43fb8df12c744884a0ff3c438a5978fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34623 } }
I20260808 13:08:56.219017 1393 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:56.219103 1393 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43fb8df12c744884a0ff3c438a5978fe, State: Initialized, Role: FOLLOWER
I20260808 13:08:56.219681 1393 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43fb8df12c744884a0ff3c438a5978fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34623 } }
I20260808 13:08:56.219938 1393 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:08:56.220110 1393 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:08:56.220329 1393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:56.222752 1393 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43fb8df12c744884a0ff3c438a5978fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34623 } }
I20260808 13:08:56.223260 1393 leader_election.cc:302] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43fb8df12c744884a0ff3c438a5978fe; no voters:
I20260808 13:08:56.223984 1393 leader_election.cc:288] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:08:56.224215 1405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:08:56.224925 1405 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe [term 1 LEADER]: Becoming Leader. State: Replica: 43fb8df12c744884a0ff3c438a5978fe, State: Running, Role: LEADER
I20260808 13:08:56.225494 1405 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43fb8df12c744884a0ff3c438a5978fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34623 } }
I20260808 13:08:56.226442 1393 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:08:56.228647 1406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43fb8df12c744884a0ff3c438a5978fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43fb8df12c744884a0ff3c438a5978fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34623 } } }
I20260808 13:08:56.228672 1407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43fb8df12c744884a0ff3c438a5978fe. Latest consensus state: current_term: 1 leader_uuid: "43fb8df12c744884a0ff3c438a5978fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43fb8df12c744884a0ff3c438a5978fe" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34623 } } }
I20260808 13:08:56.229127 1406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe [sys.catalog]: This master's current role is: LEADER
I20260808 13:08:56.229179 1407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe [sys.catalog]: This master's current role is: LEADER
I20260808 13:08:56.231285 1413 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:08:56.237350 1413 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:08:56.247128 1413 catalog_manager.cc:1347] Generated new cluster ID: 424b703ebd574a39a2de00f9d9d79042
I20260808 13:08:56.247262 1413 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:08:56.263183 1413 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:08:56.263638 1413 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:08:56.276811 1413 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43fb8df12c744884a0ff3c438a5978fe: Generated new TSK 0
I20260808 13:08:56.277730 1413 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:08:56.282076 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 1289
W20260808 13:08:56.283186 1386 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53353
W20260808 13:08:56.283908 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 1289: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:08:56.284098 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 1289
2026-08-08T13:08:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2252 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:08:56 dist-test-slave-t3q3 krb5kdc[1433](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:08:56 dist-test-slave-t3q3 krb5kdc[1433](info): set up 2 sockets
Aug 08 13:08:56 dist-test-slave-t3q3 krb5kdc[1433](info): commencing operation
krb5kdc: starting...
W20260808 13:08:57.641609 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.313s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:08:57 dist-test-slave-t3q3 krb5kdc[1433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:08:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:08:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:08:57.999821 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42561
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42561
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:08:58.355433 1457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:08:58.355715 1457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:08:58.355808 1457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:08:58.363058 1457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:08:58.363171 1457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:08:58.363220 1457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:08:58.363289 1457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:08:58.363333 1457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:08:58.373274 1457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42561
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:08:58.374557 1457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:08:58.375859 1457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:08:58.388388 1467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:58.388473 1464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:08:58.388422 1465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:08:58.393882 1457 server_base.cc:1034] running on GCE node
I20260808 13:08:58.394778 1457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:08:58.396318 1457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:08:58.397574 1457 hybrid_clock.cc:648] HybridClock initialized: now 1786194538397506 us; error 39 us; skew 500 ppm
I20260808 13:08:58.398046 1457 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:08:58 dist-test-slave-t3q3 krb5kdc[1433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194538, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:08:58.401700 1457 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:08:58.404408 1457 webserver.cc:460] Webserver started at http://127.0.0.1:41317/ using document root <none> and password file <none>
I20260808 13:08:58.405454 1457 fs_manager.cc:359] Metadata directory not provided
I20260808 13:08:58.405572 1457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:08:58.405962 1457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:08:58.409283 1457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "49e952adc9f94e26bae78552c5c694ca"
format_stamp: "Formatted at 2026-08-08 13:08:58 on dist-test-slave-t3q3"
server_key: "dc379abbdf1e06ecb5feca6d6964c0ea"
server_key_iv: "aa4cf8d2f7769397cd3941243a8cf400"
server_key_version: "encryptionkey@0"
I20260808 13:08:58.410125 1457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "49e952adc9f94e26bae78552c5c694ca"
format_stamp: "Formatted at 2026-08-08 13:08:58 on dist-test-slave-t3q3"
server_key: "dc379abbdf1e06ecb5feca6d6964c0ea"
server_key_iv: "aa4cf8d2f7769397cd3941243a8cf400"
server_key_version: "encryptionkey@0"
I20260808 13:08:58.415678 1457 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 13:08:58.419392 1479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:08:58.420914 1457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:08:58.421104 1457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "49e952adc9f94e26bae78552c5c694ca"
format_stamp: "Formatted at 2026-08-08 13:08:58 on dist-test-slave-t3q3"
server_key: "dc379abbdf1e06ecb5feca6d6964c0ea"
server_key_iv: "aa4cf8d2f7769397cd3941243a8cf400"
server_key_version: "encryptionkey@0"
I20260808 13:08:58.421320 1457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:08:58.450798 1457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:08:58.455478 1457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:08:58.455824 1457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:08:58.476495 1457 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42561
I20260808 13:08:58.476533 1561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42561 every 8 connection(s)
I20260808 13:08:58.477970 1457 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:08:58.481868 1563 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:08:58.485296 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 1457
I20260808 13:08:58.485522 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:08:58.486025 1522 external_mini_cluster.cc:1428] Setting key f61db091f5342cc69fd4e047434eeac0
I20260808 13:08:58.495049 1563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca: Bootstrap starting.
I20260808 13:08:58.499109 1563 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca: Neither blocks nor log segments found. Creating new log.
I20260808 13:08:58.500587 1563 log.cc:824] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca: Log is configured to *not* fsync() on all Append() calls
Aug 08 13:08:58 dist-test-slave-t3q3 krb5kdc[1433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194537, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 13:08:58.503978 1563 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca: No bootstrap required, opened a new log
I20260808 13:08:58.511072 1568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:58.488002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48611 (local address 127.0.0.1:42561)
0808 13:08:58.488931 (+ 929us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:58.488944 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:58.488988 (+ 44us) server_negotiation.cc:407] Connection header received
0808 13:08:58.490728 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:58.490753 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:58.491214 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:58.491571 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:58.493516 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:58.494865 (+ 1349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:58.496652 (+ 1787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:58.497354 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:58.497484 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:58.502978 (+ 5494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:58.502986 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:58.503002 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:58.503042 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:58.506499 (+ 3457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:58.507443 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:58.507450 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:58.507460 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:58.507649 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:58.508190 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:58.508196 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:58.508201 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:58.508776 (+ 575us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:58.509026 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:58.509501 (+ 475us) server_negotiation.cc:299] Negotiation successful
0808 13:08:58.509924 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":712,"thread_start_us":430,"threads_started":1}
I20260808 13:08:58.513567 1563 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e952adc9f94e26bae78552c5c694ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42561 } }
I20260808 13:08:58.514118 1563 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:08:58.514222 1563 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49e952adc9f94e26bae78552c5c694ca, State: Initialized, Role: FOLLOWER
I20260808 13:08:58.514909 1563 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e952adc9f94e26bae78552c5c694ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42561 } }
I20260808 13:08:58.515202 1563 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:08:58.515265 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:08:58.515367 1563 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:08:58.515553 1563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:08:58.517926 1563 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e952adc9f94e26bae78552c5c694ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42561 } }
I20260808 13:08:58.518558 1563 leader_election.cc:302] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49e952adc9f94e26bae78552c5c694ca; no voters:
I20260808 13:08:58.519173 1563 leader_election.cc:288] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:08:58.519313 1578 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:08:58.521533 1563 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:08:58.532385 1578 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca [term 1 LEADER]: Becoming Leader. State: Replica: 49e952adc9f94e26bae78552c5c694ca, State: Running, Role: LEADER
I20260808 13:08:58.533284 1578 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e952adc9f94e26bae78552c5c694ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42561 } }
I20260808 13:08:58.538244 1579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49e952adc9f94e26bae78552c5c694ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e952adc9f94e26bae78552c5c694ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42561 } } }
I20260808 13:08:58.539175 1579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca [sys.catalog]: This master's current role is: LEADER
I20260808 13:08:58.539623 1580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49e952adc9f94e26bae78552c5c694ca. Latest consensus state: current_term: 1 leader_uuid: "49e952adc9f94e26bae78552c5c694ca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49e952adc9f94e26bae78552c5c694ca" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42561 } } }
I20260808 13:08:58.539983 1580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca [sys.catalog]: This master's current role is: LEADER
I20260808 13:08:58.549978 1568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:08:58.520798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49823 (local address 127.0.0.1:42561)
0808 13:08:58.523943 (+ 3145us) server_negotiation.cc:206] Beginning negotiation
0808 13:08:58.523953 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:08:58.530210 (+ 6257us) server_negotiation.cc:407] Connection header received
0808 13:08:58.530722 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:08:58.530731 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:08:58.530844 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:08:58.530937 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:08:58.533473 (+ 2536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:58.535196 (+ 1723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:58.537077 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:08:58.537864 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:08:58.538752 (+ 888us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:08:58.541818 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:08:58.541826 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:08:58.541833 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:08:58.541884 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:08:58.545885 (+ 4001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:58.546830 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:58.546835 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:58.546840 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:58.547000 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:08:58.547489 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:08:58.547494 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:08:58.547497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:08:58.547932 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:08:58.549183 (+ 1251us) server_negotiation.cc:1035] Waiting for connection context
0808 13:08:58.549349 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 13:08:58.549589 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2973}
W20260808 13:08:58.550793 1595 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:08:58.550956 1595 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:08:58.551122 1599 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:08:58.553038 1599 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:08:58.561751 1599 catalog_manager.cc:1347] Generated new cluster ID: 49bca377e2fa4398b5d44b410270dcba
I20260808 13:08:58.561878 1599 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:08:58.578857 1599 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:08:58.580317 1599 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:08:58.592869 1599 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49e952adc9f94e26bae78552c5c694ca: Generated new TSK 0
I20260808 13:08:58.593834 1599 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:08:58.608034 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 1457
I20260808 13:08:58.609697 1556 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:08:58.702740 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 1457
2026-08-08T13:08:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2422 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:08:58 dist-test-slave-t3q3 krb5kdc[1606](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:08:58 dist-test-slave-t3q3 krb5kdc[1606](info): set up 2 sockets
Aug 08 13:08:58 dist-test-slave-t3q3 krb5kdc[1606](info): commencing operation
krb5kdc: starting...
W20260808 13:09:00.052980 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:09:00 dist-test-slave-t3q3 krb5kdc[1606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:09:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:09:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:09:00.387405 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46205
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32881
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46205
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:00.739319 1628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:00.739678 1628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:00.739857 1628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:00.747515 1628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:09:00.747658 1628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:00.747750 1628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:00.747866 1628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:09:00.747936 1628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:09:00.758279 1628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32881
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46205
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:00.759625 1628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:00.761006 1628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:00.773341 1640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:00.773310 1636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:00.773726 1635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:00.774457 1628 server_base.cc:1034] running on GCE node
I20260808 13:09:00.775336 1628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:00.776873 1628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:00.778117 1628 hybrid_clock.cc:648] HybridClock initialized: now 1786194540778060 us; error 43 us; skew 500 ppm
I20260808 13:09:00.778478 1628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:00 dist-test-slave-t3q3 krb5kdc[1606](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194540, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:00.781821 1628 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:09:00.783438 1628 webserver.cc:460] Webserver started at http://127.0.0.1:36241/ using document root <none> and password file <none>
I20260808 13:09:00.784210 1628 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:00.784363 1628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:00.784686 1628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:00.787725 1628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "ca945497401848ddbe32b8d46b8c1352"
format_stamp: "Formatted at 2026-08-08 13:09:00 on dist-test-slave-t3q3"
server_key: "f791e35fed8b809ac6de53e56e3b38b5"
server_key_iv: "8303bb85b906037e984178ca813bd7a4"
server_key_version: "encryptionkey@0"
I20260808 13:09:00.788559 1628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "ca945497401848ddbe32b8d46b8c1352"
format_stamp: "Formatted at 2026-08-08 13:09:00 on dist-test-slave-t3q3"
server_key: "f791e35fed8b809ac6de53e56e3b38b5"
server_key_iv: "8303bb85b906037e984178ca813bd7a4"
server_key_version: "encryptionkey@0"
I20260808 13:09:00.793936 1628 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 13:09:00.797715 1651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:00.799276 1628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:09:00.799474 1628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "ca945497401848ddbe32b8d46b8c1352"
format_stamp: "Formatted at 2026-08-08 13:09:00 on dist-test-slave-t3q3"
server_key: "f791e35fed8b809ac6de53e56e3b38b5"
server_key_iv: "8303bb85b906037e984178ca813bd7a4"
server_key_version: "encryptionkey@0"
I20260808 13:09:00.799695 1628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:00.811024 1628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:00.815310 1628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:00.815603 1628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:00.835361 1628 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46205
I20260808 13:09:00.835381 1728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46205 every 8 connection(s)
I20260808 13:09:00.836896 1628 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:09:00.840827 1729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:00.840916 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 1628
I20260808 13:09:00.841136 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:09:00.841656 1522 external_mini_cluster.cc:1428] Setting key ddbbc975c7a1aab0ecf479cf4411129f
I20260808 13:09:00.854889 1729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352: Bootstrap starting.
I20260808 13:09:00.855804 1733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:00.843475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55843 (local address 127.0.0.1:46205)
0808 13:09:00.844390 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:00.844401 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:00.844446 (+ 45us) server_negotiation.cc:407] Connection header received
0808 13:09:00.846089 (+ 1643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:00.846119 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:00.846519 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:00.846925 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:00.847517 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:00.849836 (+ 2319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:00.851229 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:00.851950 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:00.852197 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:00.853430 (+ 1233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:00.853438 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:00.853453 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:09:00.853994 (+ 541us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:00.854072 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:09:00.854098 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:00.854237 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:00.854495 (+ 258us) server_negotiation.cc:299] Negotiation successful
0808 13:09:00.854770 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":706,"thread_start_us":379,"threads_started":1}
I20260808 13:09:00.858590 1729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:00.859654 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:09:00.859863 1729 log.cc:824] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352: Log is configured to *not* fsync() on all Append() calls
I20260808 13:09:00.862838 1729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352: No bootstrap required, opened a new log
I20260808 13:09:00.871368 1733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:00.864268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57087 (local address 127.0.0.1:46205)
0808 13:09:00.864494 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:00.864502 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:00.864945 (+ 443us) server_negotiation.cc:407] Connection header received
0808 13:09:00.865136 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:00.865140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:00.865202 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:00.865287 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:00.865987 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:00.866735 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:00.868680 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:00.869154 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:00.869256 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:00.870487 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:00.870492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:00.870495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:09:00.870512 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:00.870548 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:09:00.870563 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:00.870637 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:00.870799 (+ 162us) server_negotiation.cc:299] Negotiation successful
0808 13:09:00.870879 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 13:09:00.873282 1729 raft_consensus.cc:348] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca945497401848ddbe32b8d46b8c1352" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46205 } }
I20260808 13:09:00.873715 1729 raft_consensus.cc:374] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:09:00.873833 1729 raft_consensus.cc:729] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca945497401848ddbe32b8d46b8c1352, State: Initialized, Role: FOLLOWER
I20260808 13:09:00.874428 1729 consensus_queue.cc:260] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca945497401848ddbe32b8d46b8c1352" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46205 } }
I20260808 13:09:00.874686 1729 raft_consensus.cc:388] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:09:00.874835 1729 raft_consensus.cc:482] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:09:00.875011 1729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:00.877488 1729 raft_consensus.cc:504] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca945497401848ddbe32b8d46b8c1352" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46205 } }
I20260808 13:09:00.878137 1729 leader_election.cc:302] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ca945497401848ddbe32b8d46b8c1352; no voters:
I20260808 13:09:00.878762 1729 leader_election.cc:288] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:09:00.878901 1742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:09:00.879707 1742 raft_consensus.cc:686] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352 [term 1 LEADER]: Becoming Leader. State: Replica: ca945497401848ddbe32b8d46b8c1352, State: Running, Role: LEADER
I20260808 13:09:00.880368 1742 consensus_queue.cc:237] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca945497401848ddbe32b8d46b8c1352" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46205 } }
I20260808 13:09:00.881170 1729 sys_catalog.cc:564] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:09:00.883329 1744 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ca945497401848ddbe32b8d46b8c1352" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca945497401848ddbe32b8d46b8c1352" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46205 } } }
I20260808 13:09:00.883257 1745 sys_catalog.cc:455] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ca945497401848ddbe32b8d46b8c1352. Latest consensus state: current_term: 1 leader_uuid: "ca945497401848ddbe32b8d46b8c1352" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ca945497401848ddbe32b8d46b8c1352" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46205 } } }
I20260808 13:09:00.883872 1745 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:00.883877 1744 sys_catalog.cc:458] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:00.884871 1755 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:09:00.895202 1755 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:09:00.904696 1755 catalog_manager.cc:1347] Generated new cluster ID: 16ef80f0d720499fa5ce19e2c2b81799
I20260808 13:09:00.904831 1755 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:09:00.919919 1755 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:09:00.921814 1755 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:09:00.935808 1755 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ca945497401848ddbe32b8d46b8c1352: Generated new TSK 0
I20260808 13:09:00.936750 1755 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:09:00.947907 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 1628
W20260808 13:09:00.949518 1721 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55843
W20260808 13:09:00.950400 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 1628: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:09:00.950558 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 1628
2026-08-08T13:09:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2244 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:09:01 dist-test-slave-t3q3 krb5kdc[1780](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:09:01 dist-test-slave-t3q3 krb5kdc[1780](info): set up 2 sockets
Aug 08 13:09:01 dist-test-slave-t3q3 krb5kdc[1780](info): commencing operation
krb5kdc: starting...
W20260808 13:09:02.294850 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.300s user 0.002s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:09:02 dist-test-slave-t3q3 krb5kdc[1780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:09:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:09:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:09:02.641857 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38717
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:03.004935 1800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:03.005254 1800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:03.005424 1800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:03.013001 1800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:09:03.013141 1800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:03.013191 1800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:03.013249 1800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:09:03.013288 1800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:09:03.023444 1800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:03.024837 1800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:03.026207 1800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:03.038615 1807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:03.038534 1811 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:03.038573 1808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:03.039915 1800 server_base.cc:1034] running on GCE node
I20260808 13:09:03.040820 1800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:03.042428 1800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:03.043690 1800 hybrid_clock.cc:648] HybridClock initialized: now 1786194543043635 us; error 44 us; skew 500 ppm
I20260808 13:09:03.044148 1800 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:03 dist-test-slave-t3q3 krb5kdc[1780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194543, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:03.050552 1800 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:09:03.052464 1800 webserver.cc:460] Webserver started at http://127.0.0.1:44503/ using document root <none> and password file <none>
I20260808 13:09:03.053282 1800 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:03.053453 1800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:03.053809 1800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:03.057281 1800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "15ef01f51b1240e3b1d0bfaca5a4e7ed"
format_stamp: "Formatted at 2026-08-08 13:09:03 on dist-test-slave-t3q3"
server_key: "078cce3cf7925b9abb03efb0ec560fa4"
server_key_iv: "93dd1e2951b9dd8e513251275a1a7287"
server_key_version: "encryptionkey@0"
I20260808 13:09:03.058174 1800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "15ef01f51b1240e3b1d0bfaca5a4e7ed"
format_stamp: "Formatted at 2026-08-08 13:09:03 on dist-test-slave-t3q3"
server_key: "078cce3cf7925b9abb03efb0ec560fa4"
server_key_iv: "93dd1e2951b9dd8e513251275a1a7287"
server_key_version: "encryptionkey@0"
I20260808 13:09:03.063894 1800 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 13:09:03.067777 1820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:03.069270 1800 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:09:03.069478 1800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "15ef01f51b1240e3b1d0bfaca5a4e7ed"
format_stamp: "Formatted at 2026-08-08 13:09:03 on dist-test-slave-t3q3"
server_key: "078cce3cf7925b9abb03efb0ec560fa4"
server_key_iv: "93dd1e2951b9dd8e513251275a1a7287"
server_key_version: "encryptionkey@0"
I20260808 13:09:03.069705 1800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:03.076241 1800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:03.080828 1800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:03.081161 1800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:03.102147 1800 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38717
I20260808 13:09:03.102218 1888 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38717 every 8 connection(s)
I20260808 13:09:03.103703 1800 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:09:03.106665 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 1800
I20260808 13:09:03.106868 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:09:03.107470 1522 external_mini_cluster.cc:1428] Setting key 2da6e416ddb871b09129c59ac67c258e
I20260808 13:09:03.108343 1889 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:03.115722 1892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:03.109396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38319 (local address 127.0.0.1:38717)
0808 13:09:03.110272 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:03.110285 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:03.110329 (+ 44us) server_negotiation.cc:407] Connection header received
0808 13:09:03.111997 (+ 1668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:03.112020 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:03.112466 (+ 446us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:03.112831 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:03.113984 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:03.113991 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:03.114003 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:09:03.114018 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:03.114108 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:09:03.114129 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:03.114304 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:03.114427 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 13:09:03.114663 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":617,"thread_start_us":393,"threads_started":1}
I20260808 13:09:03.119622 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:09:03.124246 1889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed: Bootstrap starting.
I20260808 13:09:03.128793 1892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:03.125998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41919 (local address 127.0.0.1:38717)
0808 13:09:03.126210 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:03.126216 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:03.126630 (+ 414us) server_negotiation.cc:407] Connection header received
0808 13:09:03.126780 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:03.126789 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:03.126855 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:03.126980 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:03.127993 (+ 1013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:03.127999 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:03.128003 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:09:03.128023 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:03.128107 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:09:03.128130 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:03.128283 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:03.128386 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 13:09:03.128448 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 13:09:03.128878 1889 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:03.130046 1889 log.cc:824] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed: Log is configured to *not* fsync() on all Append() calls
I20260808 13:09:03.133165 1889 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed: No bootstrap required, opened a new log
I20260808 13:09:03.140677 1889 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ef01f51b1240e3b1d0bfaca5a4e7ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38717 } }
I20260808 13:09:03.141173 1889 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:09:03.141301 1889 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15ef01f51b1240e3b1d0bfaca5a4e7ed, State: Initialized, Role: FOLLOWER
I20260808 13:09:03.141935 1889 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ef01f51b1240e3b1d0bfaca5a4e7ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38717 } }
I20260808 13:09:03.142200 1889 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:09:03.142362 1889 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:09:03.142550 1889 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:03.144896 1889 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ef01f51b1240e3b1d0bfaca5a4e7ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38717 } }
I20260808 13:09:03.145433 1889 leader_election.cc:302] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15ef01f51b1240e3b1d0bfaca5a4e7ed; no voters:
I20260808 13:09:03.146221 1889 leader_election.cc:288] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:09:03.146317 1907 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:09:03.147166 1907 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed [term 1 LEADER]: Becoming Leader. State: Replica: 15ef01f51b1240e3b1d0bfaca5a4e7ed, State: Running, Role: LEADER
I20260808 13:09:03.147882 1907 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ef01f51b1240e3b1d0bfaca5a4e7ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38717 } }
I20260808 13:09:03.148772 1889 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:09:03.150990 1908 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15ef01f51b1240e3b1d0bfaca5a4e7ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ef01f51b1240e3b1d0bfaca5a4e7ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38717 } } }
I20260808 13:09:03.151446 1908 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:03.151367 1909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15ef01f51b1240e3b1d0bfaca5a4e7ed. Latest consensus state: current_term: 1 leader_uuid: "15ef01f51b1240e3b1d0bfaca5a4e7ed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15ef01f51b1240e3b1d0bfaca5a4e7ed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38717 } } }
I20260808 13:09:03.151665 1909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:03.155447 1918 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:09:03.159999 1918 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:09:03.169261 1918 catalog_manager.cc:1347] Generated new cluster ID: d2da780ff93b477483ffd68f1308b593
I20260808 13:09:03.169402 1918 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:09:03.214357 1918 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:09:03.214874 1918 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:09:03.225556 1918 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15ef01f51b1240e3b1d0bfaca5a4e7ed: Generated new TSK 0
I20260808 13:09:03.226498 1918 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:09:03.242296 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 1800
W20260808 13:09:03.243538 1884 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:38319
W20260808 13:09:03.244417 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 1800: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:09:03.244589 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 1800
2026-08-08T13:09:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2295 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:09:03 dist-test-slave-t3q3 krb5kdc[1936](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:09:03 dist-test-slave-t3q3 krb5kdc[1936](info): set up 2 sockets
Aug 08 13:09:03 dist-test-slave-t3q3 krb5kdc[1936](info): commencing operation
krb5kdc: starting...
W20260808 13:09:04.609882 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.319s user 0.001s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:09:04 dist-test-slave-t3q3 krb5kdc[1936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:09:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:09:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:09:04.967159 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40745
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44653
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40745
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:05.323638 1954 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:05.323966 1954 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:05.324216 1954 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:05.331789 1954 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:09:05.331907 1954 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:05.332020 1954 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:05.332171 1954 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:09:05.332254 1954 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:09:05.343598 1954 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44653
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40745
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40745
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:05.344938 1954 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:05.346184 1954 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:05.358129 1963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:05.358124 1964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:05.358181 1966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:05.358866 1954 server_base.cc:1034] running on GCE node
I20260808 13:09:05.359618 1954 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:05.361084 1954 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:05.362319 1954 hybrid_clock.cc:648] HybridClock initialized: now 1786194545362292 us; error 85 us; skew 500 ppm
I20260808 13:09:05.362669 1954 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:05 dist-test-slave-t3q3 krb5kdc[1936](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194545, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:05.365821 1954 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:09:05.367329 1954 webserver.cc:460] Webserver started at http://127.0.0.1:43753/ using document root <none> and password file <none>
I20260808 13:09:05.368574 1954 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:05.368755 1954 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:05.369125 1954 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:05.372192 1954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "20165ab7508b4bb0b9dfb4b857eb1263"
format_stamp: "Formatted at 2026-08-08 13:09:05 on dist-test-slave-t3q3"
server_key: "7ddf88d07339e0ccfa23f79f144f3a31"
server_key_iv: "341ac0e659c686ed37fa87aa76387c0a"
server_key_version: "encryptionkey@0"
I20260808 13:09:05.372994 1954 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "20165ab7508b4bb0b9dfb4b857eb1263"
format_stamp: "Formatted at 2026-08-08 13:09:05 on dist-test-slave-t3q3"
server_key: "7ddf88d07339e0ccfa23f79f144f3a31"
server_key_iv: "341ac0e659c686ed37fa87aa76387c0a"
server_key_version: "encryptionkey@0"
I20260808 13:09:05.378346 1954 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:09:05.382107 1976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:05.383525 1954 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:09:05.383715 1954 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "20165ab7508b4bb0b9dfb4b857eb1263"
format_stamp: "Formatted at 2026-08-08 13:09:05 on dist-test-slave-t3q3"
server_key: "7ddf88d07339e0ccfa23f79f144f3a31"
server_key_iv: "341ac0e659c686ed37fa87aa76387c0a"
server_key_version: "encryptionkey@0"
I20260808 13:09:05.383932 1954 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:05.424924 1954 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:05.429241 1954 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:05.429545 1954 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:05.449728 1954 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40745
I20260808 13:09:05.449765 2051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40745 every 8 connection(s)
I20260808 13:09:05.451113 1954 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:09:05.453388 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 1954
I20260808 13:09:05.453598 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:09:05.454144 1522 external_mini_cluster.cc:1428] Setting key 57f5a2fa5913cae6d009ddb53e65101b
I20260808 13:09:05.455734 2052 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:05.467751 2052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263: Bootstrap starting.
Aug 08 13:09:05 dist-test-slave-t3q3 krb5kdc[1936](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194544, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 13:09:05.471731 2052 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:05.472918 2052 log.cc:824] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263: Log is configured to *not* fsync() on all Append() calls
I20260808 13:09:05.476636 2052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263: No bootstrap required, opened a new log
I20260808 13:09:05.477643 2058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:05.456220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:55497 (local address 127.0.0.1:40745)
0808 13:09:05.456998 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:05.457011 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:05.457052 (+ 41us) server_negotiation.cc:407] Connection header received
0808 13:09:05.458696 (+ 1644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:05.458718 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:05.459126 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:05.459561 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:05.461077 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:05.462424 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:05.463723 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:05.464481 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:05.464732 (+ 251us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:05.469587 (+ 4855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:05.469595 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:05.469613 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:05.469655 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:05.472882 (+ 3227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:05.473962 (+ 1080us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:05.473974 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:05.473985 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:05.474244 (+ 259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:05.474862 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:05.474866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:05.474867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:05.475401 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:05.475663 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:05.476247 (+ 584us) server_negotiation.cc:299] Negotiation successful
0808 13:09:05.476551 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":330,"threads_started":1}
I20260808 13:09:05.481600 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:09:05.486492 2052 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20165ab7508b4bb0b9dfb4b857eb1263" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } }
I20260808 13:09:05.486965 2052 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:09:05.487157 2052 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20165ab7508b4bb0b9dfb4b857eb1263, State: Initialized, Role: FOLLOWER
I20260808 13:09:05.487960 2052 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20165ab7508b4bb0b9dfb4b857eb1263" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } }
I20260808 13:09:05.488322 2052 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:09:05.488490 2052 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:09:05.488686 2052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:05.491811 2052 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20165ab7508b4bb0b9dfb4b857eb1263" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } }
I20260808 13:09:05.492465 2052 leader_election.cc:302] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20165ab7508b4bb0b9dfb4b857eb1263; no voters:
I20260808 13:09:05.493153 2052 leader_election.cc:288] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:09:05.493300 2067 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:09:05.494575 2067 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263 [term 1 LEADER]: Becoming Leader. State: Replica: 20165ab7508b4bb0b9dfb4b857eb1263, State: Running, Role: LEADER
I20260808 13:09:05.495244 2067 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20165ab7508b4bb0b9dfb4b857eb1263" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } }
I20260808 13:09:05.495572 2052 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:09:05.506636 2068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20165ab7508b4bb0b9dfb4b857eb1263" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20165ab7508b4bb0b9dfb4b857eb1263" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } } }
I20260808 13:09:05.507094 2068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:05.508050 2058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:05.486996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:45011 (local address 127.0.0.1:40745)
0808 13:09:05.487218 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:05.487227 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:05.487690 (+ 463us) server_negotiation.cc:407] Connection header received
0808 13:09:05.487888 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:05.487892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:05.487958 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:05.488038 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:05.490097 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:05.490915 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:05.492732 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:05.493441 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:05.493592 (+ 151us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:05.497432 (+ 3840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:05.497442 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:05.497450 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:05.497504 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:05.502499 (+ 4995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:05.503662 (+ 1163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:05.503671 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:05.503678 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:05.503909 (+ 231us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:05.504537 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:05.504545 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:05.504551 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:05.505204 (+ 653us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:05.505635 (+ 431us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:05.507210 (+ 1575us) server_negotiation.cc:299] Negotiation successful
0808 13:09:05.507480 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 13:09:05.513453 2080 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:09:05.513393 2070 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20165ab7508b4bb0b9dfb4b857eb1263. Latest consensus state: current_term: 1 leader_uuid: "20165ab7508b4bb0b9dfb4b857eb1263" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20165ab7508b4bb0b9dfb4b857eb1263" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40745 } } }
I20260808 13:09:05.513813 2070 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:05.519110 2080 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:09:05.528798 2080 catalog_manager.cc:1347] Generated new cluster ID: 8e72927410c6439a88d83d60e5d8a5ed
I20260808 13:09:05.528931 2080 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:09:05.558609 2080 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:09:05.559959 2080 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:09:05.567662 2080 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20165ab7508b4bb0b9dfb4b857eb1263: Generated new TSK 0
I20260808 13:09:05.568764 2080 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:09:05.587658 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 1954
I20260808 13:09:05.589332 2043 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:09:05.685459 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 1954
W20260808 13:09:05.711861 1930 connection.cc:535] client connection to 127.0.0.1:40745 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40745): Connection reset by peer (error 104)
2026-08-08T13:09:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2440 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:09:05 dist-test-slave-t3q3 krb5kdc[2097](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:09:05 dist-test-slave-t3q3 krb5kdc[2097](info): set up 2 sockets
Aug 08 13:09:05 dist-test-slave-t3q3 krb5kdc[2097](info): commencing operation
krb5kdc: starting...
W20260808 13:09:07.055401 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.324s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:09:07 dist-test-slave-t3q3 krb5kdc[2097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:09:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:09:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:09:07.398847 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35555
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:07.755913 2118 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:07.756325 2118 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:07.756496 2118 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:07.764014 2118 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:09:07.764209 2118 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:07.764303 2118 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:07.764412 2118 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:09:07.764477 2118 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:09:07.774748 2118 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35555
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:07.776221 2118 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:07.777571 2118 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:07.790493 2124 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:07.790400 2125 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:07.790448 2128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:07.791568 2118 server_base.cc:1034] running on GCE node
I20260808 13:09:07.792551 2118 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:07.794169 2118 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:07.795375 2118 hybrid_clock.cc:648] HybridClock initialized: now 1786194547795332 us; error 36 us; skew 500 ppm
I20260808 13:09:07.795761 2118 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:07 dist-test-slave-t3q3 krb5kdc[2097](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194547, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:07.799216 2118 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:09:07.800961 2118 webserver.cc:460] Webserver started at http://127.0.0.1:34837/ using document root <none> and password file <none>
I20260808 13:09:07.801714 2118 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:07.801864 2118 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:07.802189 2118 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:07.805431 2118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "c047b3a229b544e3b264922e06eedeed"
format_stamp: "Formatted at 2026-08-08 13:09:07 on dist-test-slave-t3q3"
server_key: "23bc83b3c7797dbeec8761f4c150b47d"
server_key_iv: "0285c6707858e84bd11bf5054cd6de5b"
server_key_version: "encryptionkey@0"
I20260808 13:09:07.806245 2118 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "c047b3a229b544e3b264922e06eedeed"
format_stamp: "Formatted at 2026-08-08 13:09:07 on dist-test-slave-t3q3"
server_key: "23bc83b3c7797dbeec8761f4c150b47d"
server_key_iv: "0285c6707858e84bd11bf5054cd6de5b"
server_key_version: "encryptionkey@0"
I20260808 13:09:07.811928 2118 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:09:07.816035 2145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:07.817656 2118 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260808 13:09:07.817865 2118 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "c047b3a229b544e3b264922e06eedeed"
format_stamp: "Formatted at 2026-08-08 13:09:07 on dist-test-slave-t3q3"
server_key: "23bc83b3c7797dbeec8761f4c150b47d"
server_key_iv: "0285c6707858e84bd11bf5054cd6de5b"
server_key_version: "encryptionkey@0"
I20260808 13:09:07.818094 2118 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:07.837193 2118 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:07.842952 2118 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:07.843397 2118 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:07.868363 2118 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35555
I20260808 13:09:07.868403 2215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35555 every 8 connection(s)
I20260808 13:09:07.870383 2118 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:09:07.873999 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 2118
I20260808 13:09:07.874184 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:09:07.874814 1522 external_mini_cluster.cc:1428] Setting key 0996a999ed535794c6ad4bdeeb7a9e57
I20260808 13:09:07.875813 2216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:07.891150 2221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:07.876836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:44939 (local address 127.0.0.1:35555)
0808 13:09:07.877771 (+ 935us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:07.877796 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:07.877847 (+ 51us) server_negotiation.cc:407] Connection header received
0808 13:09:07.880374 (+ 2527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:07.880405 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:07.880950 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:07.881424 (+ 474us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:07.882660 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:07.884767 (+ 2107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:07.886247 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:07.886942 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:07.887141 (+ 199us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:07.888512 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:07.888520 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:07.888533 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:09:07.889081 (+ 548us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:07.889175 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:09:07.889199 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:07.889371 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:07.889667 (+ 296us) server_negotiation.cc:299] Negotiation successful
0808 13:09:07.889865 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":661,"thread_start_us":397,"threads_started":1}
I20260808 13:09:07.895116 2216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed: Bootstrap starting.
I20260808 13:09:07.895315 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:09:07.898417 2216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:07.899487 2216 log.cc:824] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed: Log is configured to *not* fsync() on all Append() calls
I20260808 13:09:07.903774 2216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed: No bootstrap required, opened a new log
I20260808 13:09:07.907362 2221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:07.899988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:49195 (local address 127.0.0.1:35555)
0808 13:09:07.900251 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:07.900259 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:07.900883 (+ 624us) server_negotiation.cc:407] Connection header received
0808 13:09:07.901143 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:07.901147 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:07.901216 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:07.901309 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:07.901938 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:07.903135 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:07.904580 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:07.905253 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:07.905397 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:07.906541 (+ 1144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:07.906548 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:07.906553 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:09:07.906575 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:07.906631 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:09:07.906658 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:07.906783 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:07.906941 (+ 158us) server_negotiation.cc:299] Negotiation successful
0808 13:09:07.907011 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 13:09:07.913128 2216 raft_consensus.cc:348] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c047b3a229b544e3b264922e06eedeed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35555 } }
I20260808 13:09:07.913511 2216 raft_consensus.cc:374] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:09:07.913590 2216 raft_consensus.cc:729] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c047b3a229b544e3b264922e06eedeed, State: Initialized, Role: FOLLOWER
I20260808 13:09:07.914100 2216 consensus_queue.cc:260] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c047b3a229b544e3b264922e06eedeed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35555 } }
I20260808 13:09:07.914297 2216 raft_consensus.cc:388] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:09:07.914404 2216 raft_consensus.cc:482] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:09:07.914548 2216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:07.916826 2216 raft_consensus.cc:504] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c047b3a229b544e3b264922e06eedeed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35555 } }
I20260808 13:09:07.917284 2216 leader_election.cc:302] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c047b3a229b544e3b264922e06eedeed; no voters:
I20260808 13:09:07.917917 2216 leader_election.cc:288] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:09:07.918048 2232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:09:07.919291 2232 raft_consensus.cc:686] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed [term 1 LEADER]: Becoming Leader. State: Replica: c047b3a229b544e3b264922e06eedeed, State: Running, Role: LEADER
I20260808 13:09:07.919831 2232 consensus_queue.cc:237] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c047b3a229b544e3b264922e06eedeed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35555 } }
I20260808 13:09:07.920295 2216 sys_catalog.cc:564] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:09:07.922804 2234 sys_catalog.cc:455] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c047b3a229b544e3b264922e06eedeed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c047b3a229b544e3b264922e06eedeed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35555 } } }
I20260808 13:09:07.923280 2234 sys_catalog.cc:458] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:07.923839 2236 sys_catalog.cc:455] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed [sys.catalog]: SysCatalogTable state changed. Reason: New leader c047b3a229b544e3b264922e06eedeed. Latest consensus state: current_term: 1 leader_uuid: "c047b3a229b544e3b264922e06eedeed" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c047b3a229b544e3b264922e06eedeed" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35555 } } }
I20260808 13:09:07.924202 2236 sys_catalog.cc:458] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:07.925129 2246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:09:07.930403 2246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:09:07.941975 2246 catalog_manager.cc:1347] Generated new cluster ID: 3fcb5a733aff4108b0d8cc68af920cc5
I20260808 13:09:07.942145 2246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:09:07.952153 2246 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:09:07.954022 2246 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:09:07.961905 2246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c047b3a229b544e3b264922e06eedeed: Generated new TSK 0
I20260808 13:09:07.962757 2246 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:09:07.975183 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 2118
W20260808 13:09:07.976377 2210 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.50:44939
W20260808 13:09:07.977231 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 2118: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:09:07.977409 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 2118
W20260808 13:09:08.005084 2094 connection.cc:535] client connection to 127.0.0.1:35555 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35555): Connection reset by peer (error 104)
2026-08-08T13:09:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2293 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:09:08 dist-test-slave-t3q3 krb5kdc[2263](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:09:08 dist-test-slave-t3q3 krb5kdc[2263](info): set up 2 sockets
Aug 08 13:09:08 dist-test-slave-t3q3 krb5kdc[2263](info): commencing operation
krb5kdc: starting...
W20260808 13:09:09.369704 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.345s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:09:09 dist-test-slave-t3q3 krb5kdc[2263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:09:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:09:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:09:09.713845 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34931
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:10.066223 2286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:10.066565 2286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:10.066735 2286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:10.073961 2286 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:09:10.074119 2286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:10.074234 2286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:10.074342 2286 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:09:10.074417 2286 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:09:10.084298 2286 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:10.085616 2286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:10.086962 2286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:10.099114 2297 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:10.099295 2294 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:10.099282 2295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:10.100162 2286 server_base.cc:1034] running on GCE node
I20260808 13:09:10.100924 2286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:10.102334 2286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:10.103577 2286 hybrid_clock.cc:648] HybridClock initialized: now 1786194550103536 us; error 46 us; skew 500 ppm
I20260808 13:09:10.103933 2286 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:10 dist-test-slave-t3q3 krb5kdc[2263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194550, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:10.107254 2286 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:09:10.108851 2286 webserver.cc:460] Webserver started at http://127.0.0.1:33309/ using document root <none> and password file <none>
I20260808 13:09:10.109560 2286 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:10.109709 2286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:10.110034 2286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:10.113300 2286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "5974514e520544fab45af26f47a5e34b"
format_stamp: "Formatted at 2026-08-08 13:09:10 on dist-test-slave-t3q3"
server_key: "63d941cbee963e1d202a4790d204eae9"
server_key_iv: "49d1eda519607331b4b885895c2cd268"
server_key_version: "encryptionkey@0"
I20260808 13:09:10.114112 2286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "5974514e520544fab45af26f47a5e34b"
format_stamp: "Formatted at 2026-08-08 13:09:10 on dist-test-slave-t3q3"
server_key: "63d941cbee963e1d202a4790d204eae9"
server_key_iv: "49d1eda519607331b4b885895c2cd268"
server_key_version: "encryptionkey@0"
I20260808 13:09:10.119428 2286 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260808 13:09:10.123028 2310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:10.124445 2286 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 13:09:10.124620 2286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "5974514e520544fab45af26f47a5e34b"
format_stamp: "Formatted at 2026-08-08 13:09:10 on dist-test-slave-t3q3"
server_key: "63d941cbee963e1d202a4790d204eae9"
server_key_iv: "49d1eda519607331b4b885895c2cd268"
server_key_version: "encryptionkey@0"
I20260808 13:09:10.124825 2286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:10.130671 2286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:10.135030 2286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:10.135354 2286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:10.154771 2286 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34931
I20260808 13:09:10.154814 2387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34931 every 8 connection(s)
I20260808 13:09:10.156320 2286 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:09:10.158802 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 2286
I20260808 13:09:10.159003 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:09:10.159623 1522 external_mini_cluster.cc:1428] Setting key 49f36be1c4bc14370a006dbaf82ec0c3
I20260808 13:09:10.160712 2390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:10.168283 2393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:10.161474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:33225 (local address 127.0.0.1:34931)
0808 13:09:10.162291 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:10.162305 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:10.162360 (+ 55us) server_negotiation.cc:407] Connection header received
0808 13:09:10.164264 (+ 1904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:10.164291 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:10.164746 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:10.165198 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:10.166188 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:10.166196 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:10.166219 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:09:10.166237 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:10.166331 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:09:10.166358 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:10.166574 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:10.166714 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 13:09:10.166918 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":322,"threads_started":1}
I20260808 13:09:10.172163 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:09:10.176884 2390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b: Bootstrap starting.
I20260808 13:09:10.181056 2393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:10.178217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:35533 (local address 127.0.0.1:34931)
0808 13:09:10.178407 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:10.178414 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:10.179011 (+ 597us) server_negotiation.cc:407] Connection header received
0808 13:09:10.179224 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:10.179227 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:10.179278 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:10.179367 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:10.180407 (+ 1040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:10.180412 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:10.180415 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:09:10.180429 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:10.180467 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:09:10.180482 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:10.180589 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:10.180729 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 13:09:10.180779 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 13:09:10.181600 2390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:10.182886 2390 log.cc:824] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b: Log is configured to *not* fsync() on all Append() calls
I20260808 13:09:10.185710 2390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b: No bootstrap required, opened a new log
I20260808 13:09:10.193056 2390 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5974514e520544fab45af26f47a5e34b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34931 } }
I20260808 13:09:10.193456 2390 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:09:10.193568 2390 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5974514e520544fab45af26f47a5e34b, State: Initialized, Role: FOLLOWER
I20260808 13:09:10.194185 2390 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5974514e520544fab45af26f47a5e34b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34931 } }
I20260808 13:09:10.194429 2390 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:09:10.194585 2390 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:09:10.194754 2390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:10.197404 2390 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5974514e520544fab45af26f47a5e34b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34931 } }
I20260808 13:09:10.197914 2390 leader_election.cc:302] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5974514e520544fab45af26f47a5e34b; no voters:
I20260808 13:09:10.198489 2390 leader_election.cc:288] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:09:10.198661 2403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:09:10.199359 2403 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b [term 1 LEADER]: Becoming Leader. State: Replica: 5974514e520544fab45af26f47a5e34b, State: Running, Role: LEADER
I20260808 13:09:10.199999 2403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5974514e520544fab45af26f47a5e34b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34931 } }
I20260808 13:09:10.200618 2390 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:09:10.202821 2405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5974514e520544fab45af26f47a5e34b. Latest consensus state: current_term: 1 leader_uuid: "5974514e520544fab45af26f47a5e34b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5974514e520544fab45af26f47a5e34b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34931 } } }
I20260808 13:09:10.203255 2405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:10.203820 2404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5974514e520544fab45af26f47a5e34b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5974514e520544fab45af26f47a5e34b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34931 } } }
I20260808 13:09:10.204195 2404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:10.204414 2411 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:09:10.210507 2411 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:09:10.219867 2411 catalog_manager.cc:1347] Generated new cluster ID: 70aa53ff0c1f44ea86f417b9a5b5c1c3
I20260808 13:09:10.220009 2411 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:09:10.250666 2411 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:09:10.251222 2411 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:09:10.269071 2411 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5974514e520544fab45af26f47a5e34b: Generated new TSK 0
I20260808 13:09:10.270228 2411 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:09:10.280953 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 2286
W20260808 13:09:10.282125 2379 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.50:33225
W20260808 13:09:10.282817 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 2286: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:09:10.282963 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 2286
2026-08-08T13:09:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2301 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:09:10 dist-test-slave-t3q3 krb5kdc[2435](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:09:10 dist-test-slave-t3q3 krb5kdc[2435](info): set up 2 sockets
Aug 08 13:09:10 dist-test-slave-t3q3 krb5kdc[2435](info): commencing operation
krb5kdc: starting...
W20260808 13:09:11.631909 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.306s user 0.001s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:09:11 dist-test-slave-t3q3 krb5kdc[2435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:09:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:09:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:09:11.968240 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39761
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44007
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39761
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:12.330127 2456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:12.330502 2456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:12.330669 2456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:12.338234 2456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:09:12.338424 2456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:12.338538 2456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:12.338651 2456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:09:12.338724 2456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:09:12.349412 2456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44007
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39761
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39761
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:12.350860 2456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:12.352383 2456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:12.364818 2466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:12.364794 2470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:12.364856 2467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:12.366178 2456 server_base.cc:1034] running on GCE node
I20260808 13:09:12.367053 2456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:12.368728 2456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:12.369975 2456 hybrid_clock.cc:648] HybridClock initialized: now 1786194552369912 us; error 48 us; skew 500 ppm
I20260808 13:09:12.370370 2456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:12 dist-test-slave-t3q3 krb5kdc[2435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194552, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:12.373796 2456 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:09:12.375622 2456 webserver.cc:460] Webserver started at http://127.0.0.1:45207/ using document root <none> and password file <none>
I20260808 13:09:12.376526 2456 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:12.376679 2456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:12.377017 2456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:12.380259 2456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "f4937e1ef9fd4116802a3d8de6d129ef"
format_stamp: "Formatted at 2026-08-08 13:09:12 on dist-test-slave-t3q3"
server_key: "5b64fa098a2795a8ba9caccec6e5cdeb"
server_key_iv: "0ca8dd97a88af8dbb841feff6e41d529"
server_key_version: "encryptionkey@0"
I20260808 13:09:12.381080 2456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "f4937e1ef9fd4116802a3d8de6d129ef"
format_stamp: "Formatted at 2026-08-08 13:09:12 on dist-test-slave-t3q3"
server_key: "5b64fa098a2795a8ba9caccec6e5cdeb"
server_key_iv: "0ca8dd97a88af8dbb841feff6e41d529"
server_key_version: "encryptionkey@0"
I20260808 13:09:12.386634 2456 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260808 13:09:12.390851 2479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:12.392499 2456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260808 13:09:12.392725 2456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "f4937e1ef9fd4116802a3d8de6d129ef"
format_stamp: "Formatted at 2026-08-08 13:09:12 on dist-test-slave-t3q3"
server_key: "5b64fa098a2795a8ba9caccec6e5cdeb"
server_key_iv: "0ca8dd97a88af8dbb841feff6e41d529"
server_key_version: "encryptionkey@0"
I20260808 13:09:12.392953 2456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:12.411218 2456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:12.416118 2456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:12.416393 2456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:12.435485 2456 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39761
I20260808 13:09:12.435542 2557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39761 every 8 connection(s)
I20260808 13:09:12.436946 2456 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:09:12.440948 2559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:12.443533 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 2456
I20260808 13:09:12.443733 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:09:12.444269 1522 external_mini_cluster.cc:1428] Setting key 714ed023a00dbf8290b686e4eccfe7c1
I20260808 13:09:12.454254 2559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef: Bootstrap starting.
I20260808 13:09:12.458340 2559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:12.459738 2559 log.cc:824] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef: Log is configured to *not* fsync() on all Append() calls
Aug 08 13:09:12 dist-test-slave-t3q3 krb5kdc[2435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194551, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 13:09:12.462826 2559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef: No bootstrap required, opened a new log
I20260808 13:09:12.470266 2563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:12.446196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:53995 (local address 127.0.0.1:39761)
0808 13:09:12.447048 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:12.447064 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:12.447116 (+ 52us) server_negotiation.cc:407] Connection header received
0808 13:09:12.449526 (+ 2410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:12.449554 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:12.450087 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:12.450558 (+ 471us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:12.451977 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:12.453979 (+ 2002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:12.455400 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:12.456591 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:12.456773 (+ 182us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:12.462289 (+ 5516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:12.462298 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:12.462313 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:12.462349 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:12.465707 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:12.466559 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:12.466566 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:12.466572 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:12.466745 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:12.467313 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:12.467323 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:12.467324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:12.467891 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:12.468377 (+ 486us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:12.468910 (+ 533us) server_negotiation.cc:299] Negotiation successful
0808 13:09:12.469241 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":346,"threads_started":1}
I20260808 13:09:12.471197 2559 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4937e1ef9fd4116802a3d8de6d129ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39761 } }
I20260808 13:09:12.471643 2559 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:09:12.471771 2559 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4937e1ef9fd4116802a3d8de6d129ef, State: Initialized, Role: FOLLOWER
I20260808 13:09:12.472491 2559 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4937e1ef9fd4116802a3d8de6d129ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39761 } }
I20260808 13:09:12.472795 2559 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:09:12.472994 2559 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:09:12.473194 2559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:12.474713 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:09:12.475955 2559 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4937e1ef9fd4116802a3d8de6d129ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39761 } }
I20260808 13:09:12.476534 2559 leader_election.cc:302] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4937e1ef9fd4116802a3d8de6d129ef; no voters:
I20260808 13:09:12.477150 2559 leader_election.cc:288] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:09:12.477314 2570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:09:12.478255 2570 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef [term 1 LEADER]: Becoming Leader. State: Replica: f4937e1ef9fd4116802a3d8de6d129ef, State: Running, Role: LEADER
I20260808 13:09:12.479061 2570 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4937e1ef9fd4116802a3d8de6d129ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39761 } }
I20260808 13:09:12.481779 2570 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4937e1ef9fd4116802a3d8de6d129ef. Latest consensus state: current_term: 1 leader_uuid: "f4937e1ef9fd4116802a3d8de6d129ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4937e1ef9fd4116802a3d8de6d129ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39761 } } }
I20260808 13:09:12.482215 2570 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:12.482728 2576 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4937e1ef9fd4116802a3d8de6d129ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4937e1ef9fd4116802a3d8de6d129ef" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39761 } } }
I20260808 13:09:12.483258 2576 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:12.492280 2559 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:09:12.492520 2580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:09:12.498020 2580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:09:12.513820 2563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:12.491676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:46109 (local address 127.0.0.1:39761)
0808 13:09:12.491932 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:12.491952 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:12.492474 (+ 522us) server_negotiation.cc:407] Connection header received
0808 13:09:12.492717 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:12.492722 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:12.492801 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:12.492917 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:12.495326 (+ 2409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:12.496978 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:12.499624 (+ 2646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:12.500671 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:12.500817 (+ 146us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:12.505492 (+ 4675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:12.505503 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:12.505512 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:12.505566 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:12.509690 (+ 4124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:12.510642 (+ 952us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:12.510649 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:12.510652 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:12.510805 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:12.511421 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:12.511427 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:12.511430 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:12.511860 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:12.512255 (+ 395us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:12.513086 (+ 831us) server_negotiation.cc:299] Negotiation successful
0808 13:09:12.513352 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 13:09:12.514401 2580 catalog_manager.cc:1347] Generated new cluster ID: 89a946d0763c41f8afc2e28ab47c6828
I20260808 13:09:12.514508 2580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:09:12.524447 2580 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:09:12.525874 2580 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:09:12.534616 2580 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4937e1ef9fd4116802a3d8de6d129ef: Generated new TSK 0
I20260808 13:09:12.535549 2580 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:09:12.543419 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 2456
I20260808 13:09:12.544843 2546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:09:12.638203 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 2456
W20260808 13:09:12.662355 2433 connection.cc:535] client connection to 127.0.0.1:39761 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39761): Connection reset by peer (error 104)
2026-08-08T13:09:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2356 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:09:12 dist-test-slave-t3q3 krb5kdc[2605](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:09:12 dist-test-slave-t3q3 krb5kdc[2605](info): set up 2 sockets
Aug 08 13:09:12 dist-test-slave-t3q3 krb5kdc[2605](info): commencing operation
krb5kdc: starting...
W20260808 13:09:14.015949 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.333s user 0.000s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:09:14 dist-test-slave-t3q3 krb5kdc[2605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:09:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:09:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:09:14.364774 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33889
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:14.719461 2623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:14.719837 2623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:14.720017 2623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:14.727330 2623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:09:14.727507 2623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:14.727646 2623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:14.727757 2623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:09:14.727851 2623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:09:14.742295 2623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33889
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:14.744146 2623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:14.746344 2623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:14.759500 2632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:14.759527 2633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:14.759896 2635 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:14.761010 2623 server_base.cc:1034] running on GCE node
I20260808 13:09:14.761843 2623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:14.763322 2623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:14.764556 2623 hybrid_clock.cc:648] HybridClock initialized: now 1786194554764506 us; error 43 us; skew 500 ppm
I20260808 13:09:14.764914 2623 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:14 dist-test-slave-t3q3 krb5kdc[2605](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194554, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:14.772948 2623 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:09:14.774655 2623 webserver.cc:460] Webserver started at http://127.0.0.1:37267/ using document root <none> and password file <none>
I20260808 13:09:14.775348 2623 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:14.775453 2623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:14.775727 2623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:14.779232 2623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "a54a30c763fd4511b75cd29a7fbb4478"
format_stamp: "Formatted at 2026-08-08 13:09:14 on dist-test-slave-t3q3"
server_key: "3cd5d9b7ed8e51fb1cb167f81401f825"
server_key_iv: "ce36d5425aa318e6c463c31981cd19e1"
server_key_version: "encryptionkey@0"
I20260808 13:09:14.780037 2623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "a54a30c763fd4511b75cd29a7fbb4478"
format_stamp: "Formatted at 2026-08-08 13:09:14 on dist-test-slave-t3q3"
server_key: "3cd5d9b7ed8e51fb1cb167f81401f825"
server_key_iv: "ce36d5425aa318e6c463c31981cd19e1"
server_key_version: "encryptionkey@0"
I20260808 13:09:14.786602 2623 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260808 13:09:14.790737 2646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:14.792383 2623 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260808 13:09:14.792609 2623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "a54a30c763fd4511b75cd29a7fbb4478"
format_stamp: "Formatted at 2026-08-08 13:09:14 on dist-test-slave-t3q3"
server_key: "3cd5d9b7ed8e51fb1cb167f81401f825"
server_key_iv: "ce36d5425aa318e6c463c31981cd19e1"
server_key_version: "encryptionkey@0"
I20260808 13:09:14.792843 2623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:14.815609 2623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:14.820999 2623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:14.821368 2623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:14.843163 2623 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33889
I20260808 13:09:14.843215 2727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33889 every 8 connection(s)
I20260808 13:09:14.844662 2623 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:09:14.848835 2728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:14.850690 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 2623
I20260808 13:09:14.850895 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:09:14.851382 1522 external_mini_cluster.cc:1428] Setting key 16fff39dc7a47bd1369b4dd23e2bd20f
I20260808 13:09:14.863006 2728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478: Bootstrap starting.
I20260808 13:09:14.864892 2732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:14.853449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:38457 (local address 127.0.0.1:33889)
0808 13:09:14.854283 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:14.854295 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:14.854336 (+ 41us) server_negotiation.cc:407] Connection header received
0808 13:09:14.855996 (+ 1660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:14.856020 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:14.856494 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:14.856913 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:14.857453 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:14.858741 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:14.860152 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:14.860887 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:14.861095 (+ 208us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:14.862457 (+ 1362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:14.862465 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:14.862478 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:09:14.863026 (+ 548us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:14.863144 (+ 118us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:09:14.863188 (+ 44us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:14.863374 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:14.863681 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 13:09:14.863852 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":612,"thread_start_us":374,"threads_started":1}
I20260808 13:09:14.866762 2728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:14.868221 2728 log.cc:824] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478: Log is configured to *not* fsync() on all Append() calls
I20260808 13:09:14.868968 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:09:14.871346 2728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478: No bootstrap required, opened a new log
I20260808 13:09:14.880615 2732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:14.873603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:52583 (local address 127.0.0.1:33889)
0808 13:09:14.873806 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:14.873814 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:14.874379 (+ 565us) server_negotiation.cc:407] Connection header received
0808 13:09:14.874614 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:14.874623 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:14.874692 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:14.874818 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:14.875337 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:14.876247 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:14.877815 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:14.878335 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:14.878519 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:14.879756 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:14.879771 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:14.879774 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:09:14.879795 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:14.879848 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:09:14.879863 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:14.879999 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:14.880270 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 13:09:14.880325 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 13:09:14.880831 2728 raft_consensus.cc:348] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54a30c763fd4511b75cd29a7fbb4478" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33889 } }
I20260808 13:09:14.881371 2728 raft_consensus.cc:374] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:09:14.881745 2728 raft_consensus.cc:729] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a54a30c763fd4511b75cd29a7fbb4478, State: Initialized, Role: FOLLOWER
I20260808 13:09:14.882555 2728 consensus_queue.cc:260] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54a30c763fd4511b75cd29a7fbb4478" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33889 } }
I20260808 13:09:14.882872 2728 raft_consensus.cc:388] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:09:14.883021 2728 raft_consensus.cc:482] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:09:14.883193 2728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:14.885910 2728 raft_consensus.cc:504] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54a30c763fd4511b75cd29a7fbb4478" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33889 } }
I20260808 13:09:14.886686 2728 leader_election.cc:302] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a54a30c763fd4511b75cd29a7fbb4478; no voters:
I20260808 13:09:14.887311 2728 leader_election.cc:288] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:09:14.887490 2739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:09:14.888232 2739 raft_consensus.cc:686] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478 [term 1 LEADER]: Becoming Leader. State: Replica: a54a30c763fd4511b75cd29a7fbb4478, State: Running, Role: LEADER
I20260808 13:09:14.888767 2739 consensus_queue.cc:237] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54a30c763fd4511b75cd29a7fbb4478" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33889 } }
I20260808 13:09:14.889604 2728 sys_catalog.cc:564] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:09:14.891507 2740 sys_catalog.cc:455] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a54a30c763fd4511b75cd29a7fbb4478" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54a30c763fd4511b75cd29a7fbb4478" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33889 } } }
I20260808 13:09:14.892168 2741 sys_catalog.cc:455] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a54a30c763fd4511b75cd29a7fbb4478. Latest consensus state: current_term: 1 leader_uuid: "a54a30c763fd4511b75cd29a7fbb4478" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a54a30c763fd4511b75cd29a7fbb4478" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33889 } } }
I20260808 13:09:14.895176 2740 sys_catalog.cc:458] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:14.895437 2741 sys_catalog.cc:458] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:14.898716 2748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:09:14.904284 2748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:09:14.914839 2748 catalog_manager.cc:1347] Generated new cluster ID: 9e0b7a5b1e644e6394992a861d528f58
I20260808 13:09:14.914997 2748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:09:14.940164 2748 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:09:14.942303 2748 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:09:14.954320 2748 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a54a30c763fd4511b75cd29a7fbb4478: Generated new TSK 0
I20260808 13:09:14.955205 2748 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:09:14.977829 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 2623
W20260808 13:09:14.979525 2722 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.50:38457
W20260808 13:09:14.980461 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 2623: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:09:14.980623 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 2623
W20260808 13:09:15.007660 2602 connection.cc:535] client connection to 127.0.0.1:33889 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33889): Connection reset by peer (error 104)
2026-08-08T13:09:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2344 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:09:15 dist-test-slave-t3q3 krb5kdc[2768](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:09:15 dist-test-slave-t3q3 krb5kdc[2768](info): set up 2 sockets
Aug 08 13:09:15 dist-test-slave-t3q3 krb5kdc[2768](info): commencing operation
krb5kdc: starting...
W20260808 13:09:16.349340 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.322s user 0.004s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:09:16 dist-test-slave-t3q3 krb5kdc[2768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:09:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:09:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 13:09:16.697834 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43507
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43507
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:17.056972 2786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:17.057264 2786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:17.057516 2786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:17.064707 2786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:09:17.064805 2786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:17.064930 2786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:17.065047 2786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:09:17.065136 2786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:09:17.074723 2786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43507
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43507
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:17.076007 2786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:17.077351 2786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:17.089902 2792 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:17.089947 2796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:17.089972 2793 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:17.090812 2786 server_base.cc:1034] running on GCE node
I20260808 13:09:17.091786 2786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:17.093351 2786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:17.094630 2786 hybrid_clock.cc:648] HybridClock initialized: now 1786194557094589 us; error 58 us; skew 500 ppm
I20260808 13:09:17.095032 2786 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:17 dist-test-slave-t3q3 krb5kdc[2768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194557, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:17.098433 2786 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 13:09:17.100163 2786 webserver.cc:460] Webserver started at http://127.0.0.1:36127/ using document root <none> and password file <none>
I20260808 13:09:17.100878 2786 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:17.101040 2786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:17.101440 2786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:17.104643 2786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "00cc4a9c377c4c63aedf58dd40a67a30"
format_stamp: "Formatted at 2026-08-08 13:09:17 on dist-test-slave-t3q3"
server_key: "cb6bcacb893221e1433970607082bdf7"
server_key_iv: "bd9d6d4675aa8945e6087c6f5522b182"
server_key_version: "encryptionkey@0"
I20260808 13:09:17.105474 2786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "00cc4a9c377c4c63aedf58dd40a67a30"
format_stamp: "Formatted at 2026-08-08 13:09:17 on dist-test-slave-t3q3"
server_key: "cb6bcacb893221e1433970607082bdf7"
server_key_iv: "bd9d6d4675aa8945e6087c6f5522b182"
server_key_version: "encryptionkey@0"
I20260808 13:09:17.110893 2786 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260808 13:09:17.114730 2808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:17.116271 2786 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:09:17.116461 2786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "00cc4a9c377c4c63aedf58dd40a67a30"
format_stamp: "Formatted at 2026-08-08 13:09:17 on dist-test-slave-t3q3"
server_key: "cb6bcacb893221e1433970607082bdf7"
server_key_iv: "bd9d6d4675aa8945e6087c6f5522b182"
server_key_version: "encryptionkey@0"
I20260808 13:09:17.116688 2786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:17.122562 2786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:17.126966 2786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:17.127331 2786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:17.147037 2786 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43507
I20260808 13:09:17.147075 2889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43507 every 8 connection(s)
I20260808 13:09:17.148617 2786 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:09:17.152626 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 2786
I20260808 13:09:17.152765 2890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:17.152825 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:09:17.153286 1522 external_mini_cluster.cc:1428] Setting key e141e0e1a3180bcb69135a4a5aa897dd
I20260808 13:09:17.162478 2898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:17.155158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:33845 (local address 127.0.0.1:43507)
0808 13:09:17.156269 (+ 1111us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:17.156281 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:17.156848 (+ 567us) server_negotiation.cc:407] Connection header received
0808 13:09:17.158689 (+ 1841us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:17.158713 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:17.159094 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:17.159561 (+ 467us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:17.160768 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:17.160775 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:17.160792 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:09:17.160806 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:17.160866 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:09:17.160888 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:17.161154 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:17.161296 (+ 142us) server_negotiation.cc:299] Negotiation successful
0808 13:09:17.161563 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":755,"thread_start_us":493,"threads_started":1}
I20260808 13:09:17.166586 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:09:17.167896 2890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30: Bootstrap starting.
I20260808 13:09:17.171418 2890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:17.172575 2890 log.cc:824] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30: Log is configured to *not* fsync() on all Append() calls
I20260808 13:09:17.173928 2898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:17.171277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.50:58575 (local address 127.0.0.1:43507)
0808 13:09:17.171498 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:17.171505 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:17.171832 (+ 327us) server_negotiation.cc:407] Connection header received
0808 13:09:17.172014 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:17.172020 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:17.172129 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:17.172243 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:17.173240 (+ 997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:17.173246 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:17.173250 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:09:17.173272 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:17.173310 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:09:17.173329 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:17.173465 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:17.173580 (+ 115us) server_negotiation.cc:299] Negotiation successful
0808 13:09:17.173643 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 13:09:17.175825 2890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30: No bootstrap required, opened a new log
I20260808 13:09:17.183665 2890 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00cc4a9c377c4c63aedf58dd40a67a30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43507 } }
I20260808 13:09:17.184103 2890 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:09:17.184213 2890 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00cc4a9c377c4c63aedf58dd40a67a30, State: Initialized, Role: FOLLOWER
I20260808 13:09:17.184844 2890 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00cc4a9c377c4c63aedf58dd40a67a30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43507 } }
I20260808 13:09:17.185106 2890 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:09:17.185279 2890 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:09:17.185467 2890 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:17.187798 2890 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00cc4a9c377c4c63aedf58dd40a67a30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43507 } }
I20260808 13:09:17.188378 2890 leader_election.cc:302] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00cc4a9c377c4c63aedf58dd40a67a30; no voters:
I20260808 13:09:17.189047 2890 leader_election.cc:288] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:09:17.189224 2909 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:09:17.189945 2909 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30 [term 1 LEADER]: Becoming Leader. State: Replica: 00cc4a9c377c4c63aedf58dd40a67a30, State: Running, Role: LEADER
I20260808 13:09:17.190541 2909 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00cc4a9c377c4c63aedf58dd40a67a30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43507 } }
I20260808 13:09:17.191259 2890 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:09:17.193740 2910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00cc4a9c377c4c63aedf58dd40a67a30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00cc4a9c377c4c63aedf58dd40a67a30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43507 } } }
I20260808 13:09:17.194311 2911 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00cc4a9c377c4c63aedf58dd40a67a30. Latest consensus state: current_term: 1 leader_uuid: "00cc4a9c377c4c63aedf58dd40a67a30" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00cc4a9c377c4c63aedf58dd40a67a30" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43507 } } }
I20260808 13:09:17.194742 2910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:17.194751 2911 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:17.195686 2919 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:09:17.201696 2919 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:09:17.211261 2919 catalog_manager.cc:1347] Generated new cluster ID: aa456359b429403ca283ea7721e77858
I20260808 13:09:17.211409 2919 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:09:17.221282 2919 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:09:17.221817 2919 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:09:17.237231 2919 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00cc4a9c377c4c63aedf58dd40a67a30: Generated new TSK 0
I20260808 13:09:17.238161 2919 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:09:17.250612 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 2786
W20260808 13:09:17.251854 2882 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.50:33845
W20260808 13:09:17.252637 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 2786: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:09:17.252789 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 2786
2026-08-08T13:09:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2271 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:09:17 dist-test-slave-t3q3 krb5kdc[2939](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:09:17 dist-test-slave-t3q3 krb5kdc[2939](info): set up 2 sockets
Aug 08 13:09:17 dist-test-slave-t3q3 krb5kdc[2939](info): commencing operation
krb5kdc: starting...
W20260808 13:09:18.618870 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.320s user 0.000s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:09:18 dist-test-slave-t3q3 krb5kdc[2939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:09:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:09:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:09:18.960533 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:38861
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:35011
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:38861
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:19.314622 2955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:19.314922 2955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:19.315121 2955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:19.322317 2955 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:09:19.322441 2955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:19.322568 2955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:19.322691 2955 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:09:19.322781 2955 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:09:19.332688 2955 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:35011
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:38861
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:38861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:19.334127 2955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:19.335525 2955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:19.348191 2965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:19.348321 2962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:19.348212 2963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:19.348943 2955 server_base.cc:1034] running on GCE node
I20260808 13:09:19.349850 2955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:19.351341 2955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:19.352617 2955 hybrid_clock.cc:648] HybridClock initialized: now 1786194559352571 us; error 59 us; skew 500 ppm
I20260808 13:09:19.353220 2955 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:19 dist-test-slave-t3q3 krb5kdc[2939](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194559, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:19.357012 2955 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:09:19.358745 2955 webserver.cc:460] Webserver started at http://127.1.124.190:39579/ using document root <none> and password file <none>
I20260808 13:09:19.359500 2955 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:19.359674 2955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:19.360028 2955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:19.363279 2955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "5642fde1afed4383a798d89c0cf629a6"
format_stamp: "Formatted at 2026-08-08 13:09:19 on dist-test-slave-t3q3"
server_key: "04ce839516976321ac228ccd23d5e63d"
server_key_iv: "5f1288d7013075fc557f142635ad5391"
server_key_version: "encryptionkey@0"
I20260808 13:09:19.364231 2955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "5642fde1afed4383a798d89c0cf629a6"
format_stamp: "Formatted at 2026-08-08 13:09:19 on dist-test-slave-t3q3"
server_key: "04ce839516976321ac228ccd23d5e63d"
server_key_iv: "5f1288d7013075fc557f142635ad5391"
server_key_version: "encryptionkey@0"
I20260808 13:09:19.369808 2955 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260808 13:09:19.373594 2982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:19.375061 2955 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:09:19.375295 2955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "5642fde1afed4383a798d89c0cf629a6"
format_stamp: "Formatted at 2026-08-08 13:09:19 on dist-test-slave-t3q3"
server_key: "04ce839516976321ac228ccd23d5e63d"
server_key_iv: "5f1288d7013075fc557f142635ad5391"
server_key_version: "encryptionkey@0"
I20260808 13:09:19.375525 2955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:19.400489 2955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:19.405414 2955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:19.405789 2955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:19.425647 2955 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:38861
I20260808 13:09:19.425665 3064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:38861 every 8 connection(s)
I20260808 13:09:19.427134 2955 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:09:19.431226 3067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:19.436961 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 2955
I20260808 13:09:19.437160 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:09:19.437759 1522 external_mini_cluster.cc:1428] Setting key 2ee4a9bf3cbd490b8608a6e709ffcc17
I20260808 13:09:19.443305 3067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6: Bootstrap starting.
I20260808 13:09:19.447288 3067 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:19.448685 3067 log.cc:824] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6: Log is configured to *not* fsync() on all Append() calls
I20260808 13:09:19.452414 3067 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6: No bootstrap required, opened a new log
Aug 08 13:09:19 dist-test-slave-t3q3 krb5kdc[2939](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194558, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:09:19.460956 3067 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5642fde1afed4383a798d89c0cf629a6" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38861 } }
I20260808 13:09:19.461422 3067 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:09:19.461586 3067 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5642fde1afed4383a798d89c0cf629a6, State: Initialized, Role: FOLLOWER
I20260808 13:09:19.462216 3067 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5642fde1afed4383a798d89c0cf629a6" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38861 } }
I20260808 13:09:19.462507 3067 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:09:19.462683 3067 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:09:19.462883 3067 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:19.464598 3070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:19.439872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45883 (local address 127.1.124.190:38861)
0808 13:09:19.441086 (+ 1214us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:19.441100 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:19.441153 (+ 53us) server_negotiation.cc:407] Connection header received
0808 13:09:19.443063 (+ 1910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:19.443087 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:19.443623 (+ 536us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:19.444030 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:19.446057 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:19.447688 (+ 1631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:19.449708 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:19.450745 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:19.450903 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:19.456273 (+ 5370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:19.456285 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:19.456303 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:19.456356 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:19.459782 (+ 3426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:19.460718 (+ 936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:19.460725 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:19.460733 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:19.460905 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:19.461429 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:19.461433 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:19.461435 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:19.462173 (+ 738us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:19.462519 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:19.463070 (+ 551us) server_negotiation.cc:299] Negotiation successful
0808 13:09:19.463411 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":960,"thread_start_us":681,"threads_started":1}
I20260808 13:09:19.465688 3067 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5642fde1afed4383a798d89c0cf629a6" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38861 } }
I20260808 13:09:19.466238 3067 leader_election.cc:302] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5642fde1afed4383a798d89c0cf629a6; no voters:
I20260808 13:09:19.467006 3067 leader_election.cc:288] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:09:19.467136 3075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:09:19.468638 3075 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6 [term 1 LEADER]: Becoming Leader. State: Replica: 5642fde1afed4383a798d89c0cf629a6, State: Running, Role: LEADER
I20260808 13:09:19.469075 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:09:19.469411 3067 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:09:19.469336 3075 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5642fde1afed4383a798d89c0cf629a6" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38861 } }
I20260808 13:09:19.474628 3079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5642fde1afed4383a798d89c0cf629a6. Latest consensus state: current_term: 1 leader_uuid: "5642fde1afed4383a798d89c0cf629a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5642fde1afed4383a798d89c0cf629a6" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38861 } } }
I20260808 13:09:19.474610 3078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5642fde1afed4383a798d89c0cf629a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5642fde1afed4383a798d89c0cf629a6" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38861 } } }
I20260808 13:09:19.475214 3079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:19.475222 3078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:19.476397 3094 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:09:19.499086 3094 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:09:19.527992 3094 catalog_manager.cc:1347] Generated new cluster ID: ef0371c606594432818a5e7c625e4c1b
I20260808 13:09:19.528158 3094 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:09:19.529093 3070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:19.508273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36257 (local address 127.1.124.190:38861)
0808 13:09:19.508663 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:19.508673 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:19.509016 (+ 343us) server_negotiation.cc:407] Connection header received
0808 13:09:19.509339 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:19.509346 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:19.509431 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:19.509511 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:19.511934 (+ 2423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:19.513445 (+ 1511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:19.515522 (+ 2077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:19.516299 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:19.516428 (+ 129us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:19.520400 (+ 3972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:19.520412 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:19.520423 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:19.520466 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:19.524795 (+ 4329us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:19.525948 (+ 1153us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:19.525954 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:19.525959 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:19.526141 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:19.526847 (+ 706us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:19.526859 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:19.526863 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:19.527319 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:19.527613 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:19.528475 (+ 862us) server_negotiation.cc:299] Negotiation successful
0808 13:09:19.528702 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226}
I20260808 13:09:19.543578 3094 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:09:19.545049 3094 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:09:19.551645 3094 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5642fde1afed4383a798d89c0cf629a6: Generated new TSK 0
I20260808 13:09:19.552513 3094 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:09:19.560511 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 2955
I20260808 13:09:19.561646 3058 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:09:19.652812 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 2955
W20260808 13:09:19.676105 2937 connection.cc:535] client connection to 127.1.124.190:38861 recv error: Network error: failed to read from TLS socket (remote: 127.1.124.190:38861): Connection reset by peer (error 104)
2026-08-08T13:09:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2396 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:09:19 dist-test-slave-t3q3 krb5kdc[3114](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:09:19 dist-test-slave-t3q3 krb5kdc[3114](info): set up 2 sockets
Aug 08 13:09:19 dist-test-slave-t3q3 krb5kdc[3114](info): commencing operation
krb5kdc: starting...
W20260808 13:09:21.012204 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.316s user 0.005s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:09:21 dist-test-slave-t3q3 krb5kdc[3114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:09:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:09:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:09:21.368911 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:37737
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:38451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:37737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:21.724579 3131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:21.724963 3131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:21.725157 3131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:21.732630 3131 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:09:21.732816 3131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:21.732936 3131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:21.733088 3131 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:09:21.733180 3131 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:09:21.743309 3131 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:38451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:37737
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:37737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:21.744772 3131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:21.746165 3131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:21.759224 3137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:21.759200 3142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:21.759260 3140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:21.760535 3131 server_base.cc:1034] running on GCE node
I20260808 13:09:21.761473 3131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:21.762966 3131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:21.764170 3131 hybrid_clock.cc:648] HybridClock initialized: now 1786194561764128 us; error 31 us; skew 500 ppm
I20260808 13:09:21.764564 3131 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:21 dist-test-slave-t3q3 krb5kdc[3114](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194561, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:21.772863 3131 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:09:21.774693 3131 webserver.cc:460] Webserver started at http://127.1.124.190:36297/ using document root <none> and password file <none>
I20260808 13:09:21.775456 3131 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:21.775619 3131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:21.775951 3131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:21.779567 3131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "8ec706531a9641a88ed4d461d88ee946"
format_stamp: "Formatted at 2026-08-08 13:09:21 on dist-test-slave-t3q3"
server_key: "4048a8c62028f2d1c53a4d45231e5227"
server_key_iv: "a505d00ea4dae8654ea35ddc22599b27"
server_key_version: "encryptionkey@0"
I20260808 13:09:21.780505 3131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "8ec706531a9641a88ed4d461d88ee946"
format_stamp: "Formatted at 2026-08-08 13:09:21 on dist-test-slave-t3q3"
server_key: "4048a8c62028f2d1c53a4d45231e5227"
server_key_iv: "a505d00ea4dae8654ea35ddc22599b27"
server_key_version: "encryptionkey@0"
I20260808 13:09:21.786463 3131 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260808 13:09:21.790562 3150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:21.792039 3131 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260808 13:09:21.792341 3131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "8ec706531a9641a88ed4d461d88ee946"
format_stamp: "Formatted at 2026-08-08 13:09:21 on dist-test-slave-t3q3"
server_key: "4048a8c62028f2d1c53a4d45231e5227"
server_key_iv: "a505d00ea4dae8654ea35ddc22599b27"
server_key_version: "encryptionkey@0"
I20260808 13:09:21.792570 3131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:21.805485 3131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:21.810328 3131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:21.810672 3131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:21.830847 3131 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:37737
I20260808 13:09:21.830870 3226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:37737 every 8 connection(s)
I20260808 13:09:21.832371 3131 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:09:21.835870 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 3131
I20260808 13:09:21.836113 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:09:21.836509 3227 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:21.836637 1522 external_mini_cluster.cc:1428] Setting key 6a6282ec0a02d8fbef10676f0934780d
I20260808 13:09:21.851403 3230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:21.838450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35381 (local address 127.1.124.190:37737)
0808 13:09:21.839258 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:21.839270 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:21.839319 (+ 49us) server_negotiation.cc:407] Connection header received
0808 13:09:21.841367 (+ 2048us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:21.841391 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:21.841796 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:21.842132 (+ 336us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:21.842838 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:21.844380 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:21.846330 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:21.847063 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:21.847250 (+ 187us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:21.848802 (+ 1552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:21.848811 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:21.848836 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:09:21.849439 (+ 603us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:21.849530 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:09:21.849560 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:21.849709 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:21.850156 (+ 447us) server_negotiation.cc:299] Negotiation successful
0808 13:09:21.850361 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":375,"threads_started":1}
I20260808 13:09:21.851434 3227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946: Bootstrap starting.
I20260808 13:09:21.855414 3227 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:21.855963 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:09:21.856712 3227 log.cc:824] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946: Log is configured to *not* fsync() on all Append() calls
I20260808 13:09:21.859885 3227 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946: No bootstrap required, opened a new log
I20260808 13:09:21.868806 3230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:21.861096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39433 (local address 127.1.124.190:37737)
0808 13:09:21.861291 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:21.861298 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:21.861879 (+ 581us) server_negotiation.cc:407] Connection header received
0808 13:09:21.862039 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:21.862045 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:21.862139 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:21.862249 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:21.862758 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:21.863977 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:21.865462 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:21.866224 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:21.866368 (+ 144us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:21.867890 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:21.867898 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:21.867904 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:09:21.867927 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:21.867993 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:09:21.868015 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:21.868185 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:21.868368 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 13:09:21.868430 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 13:09:21.869911 3227 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec706531a9641a88ed4d461d88ee946" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37737 } }
I20260808 13:09:21.870365 3227 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:09:21.870471 3227 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ec706531a9641a88ed4d461d88ee946, State: Initialized, Role: FOLLOWER
I20260808 13:09:21.871097 3227 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec706531a9641a88ed4d461d88ee946" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37737 } }
I20260808 13:09:21.871354 3227 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:09:21.871526 3227 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:09:21.871701 3227 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:21.874161 3227 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec706531a9641a88ed4d461d88ee946" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37737 } }
I20260808 13:09:21.874730 3227 leader_election.cc:302] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ec706531a9641a88ed4d461d88ee946; no voters:
I20260808 13:09:21.875452 3227 leader_election.cc:288] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:09:21.875604 3239 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:09:21.877705 3227 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:09:21.878029 3239 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946 [term 1 LEADER]: Becoming Leader. State: Replica: 8ec706531a9641a88ed4d461d88ee946, State: Running, Role: LEADER
I20260808 13:09:21.878573 3239 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec706531a9641a88ed4d461d88ee946" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37737 } }
I20260808 13:09:21.882018 3240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ec706531a9641a88ed4d461d88ee946" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec706531a9641a88ed4d461d88ee946" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37737 } } }
I20260808 13:09:21.882077 3242 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ec706531a9641a88ed4d461d88ee946. Latest consensus state: current_term: 1 leader_uuid: "8ec706531a9641a88ed4d461d88ee946" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ec706531a9641a88ed4d461d88ee946" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 37737 } } }
I20260808 13:09:21.882880 3240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:21.882877 3242 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:21.883828 3252 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:09:21.889598 3252 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:09:21.899833 3252 catalog_manager.cc:1347] Generated new cluster ID: 710af0325c4d4501936ce7243fa1fb93
I20260808 13:09:21.900086 3252 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:09:21.918365 3252 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:09:21.920488 3252 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:09:21.931998 3252 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ec706531a9641a88ed4d461d88ee946: Generated new TSK 0
I20260808 13:09:21.933324 3252 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:09:21.950012 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 3131
W20260808 13:09:21.951607 3219 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35381
W20260808 13:09:21.952615 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 3131: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:09:21.952781 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 3131
W20260808 13:09:21.980415 3112 connection.cc:535] client connection to 127.1.124.190:37737 recv error: Network error: failed to read from TLS socket (remote: 127.1.124.190:37737): Connection reset by peer (error 104)
2026-08-08T13:09:21Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2304 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:09:22 dist-test-slave-t3q3 krb5kdc[3269](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:09:22 dist-test-slave-t3q3 krb5kdc[3269](info): set up 2 sockets
Aug 08 13:09:22 dist-test-slave-t3q3 krb5kdc[3269](info): commencing operation
krb5kdc: starting...
W20260808 13:09:23.332810 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.333s user 0.009s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:09:23 dist-test-slave-t3q3 krb5kdc[3269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:09:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:09:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:09:23.688448 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:35115
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:46675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:35115
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:24.036685 3290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:24.037082 3290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:24.037262 3290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:24.044569 3290 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:09:24.044759 3290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:24.044878 3290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:24.044996 3290 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:09:24.045137 3290 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:09:24.055370 3290 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:46675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:35115
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:35115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:24.056846 3290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:24.058225 3290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:24.070822 3296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:24.070876 3302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:24.070891 3298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:24.073753 3290 server_base.cc:1034] running on GCE node
I20260808 13:09:24.074604 3290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:24.076222 3290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:24.077476 3290 hybrid_clock.cc:648] HybridClock initialized: now 1786194564077406 us; error 65 us; skew 500 ppm
I20260808 13:09:24.077886 3290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:24 dist-test-slave-t3q3 krb5kdc[3269](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194564, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:24.081341 3290 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:09:24.083009 3290 webserver.cc:460] Webserver started at http://127.1.124.190:34863/ using document root <none> and password file <none>
I20260808 13:09:24.083750 3290 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:24.083906 3290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:24.084300 3290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:24.087370 3290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "7218ff0fd7ad4f9a9dd1a27d1f381892"
format_stamp: "Formatted at 2026-08-08 13:09:24 on dist-test-slave-t3q3"
server_key: "a9a3be86dfbcd57fe768f00e3a7b06b6"
server_key_iv: "7f84b01ad90f538f3cb7895cda8cac6a"
server_key_version: "encryptionkey@0"
I20260808 13:09:24.088272 3290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "7218ff0fd7ad4f9a9dd1a27d1f381892"
format_stamp: "Formatted at 2026-08-08 13:09:24 on dist-test-slave-t3q3"
server_key: "a9a3be86dfbcd57fe768f00e3a7b06b6"
server_key_iv: "7f84b01ad90f538f3cb7895cda8cac6a"
server_key_version: "encryptionkey@0"
I20260808 13:09:24.093667 3290 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260808 13:09:24.097673 3312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:24.099242 3290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:09:24.099453 3290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "7218ff0fd7ad4f9a9dd1a27d1f381892"
format_stamp: "Formatted at 2026-08-08 13:09:24 on dist-test-slave-t3q3"
server_key: "a9a3be86dfbcd57fe768f00e3a7b06b6"
server_key_iv: "7f84b01ad90f538f3cb7895cda8cac6a"
server_key_version: "encryptionkey@0"
I20260808 13:09:24.099676 3290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:24.105854 3290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:24.110164 3290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:24.110489 3290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:24.130867 3290 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:35115
I20260808 13:09:24.130913 3382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:35115 every 8 connection(s)
I20260808 13:09:24.132397 3290 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:09:24.133955 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 3290
I20260808 13:09:24.134169 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:09:24.134822 1522 external_mini_cluster.cc:1428] Setting key 838994acf596ff55cd42da2410512c9c
I20260808 13:09:24.137825 3385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:24.143453 3389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:24.136903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35517 (local address 127.1.124.190:35115)
0808 13:09:24.137822 (+ 919us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:24.137834 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:24.137947 (+ 113us) server_negotiation.cc:407] Connection header received
0808 13:09:24.139713 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:24.139736 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:24.140168 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:24.140603 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:24.141755 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:24.141762 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:24.141774 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:09:24.141788 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:24.141873 (+ 85us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:09:24.141893 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:24.142163 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:24.142316 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 13:09:24.142500 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":673,"thread_start_us":435,"threads_started":1}
I20260808 13:09:24.147505 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:09:24.151054 3385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892: Bootstrap starting.
I20260808 13:09:24.154949 3389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:24.152128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47691 (local address 127.1.124.190:35115)
0808 13:09:24.152320 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:24.152327 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:24.152812 (+ 485us) server_negotiation.cc:407] Connection header received
0808 13:09:24.152956 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:24.152982 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:24.153050 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:24.153180 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:24.154253 (+ 1073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:24.154261 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:24.154266 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:09:24.154285 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:24.154334 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:09:24.154357 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:24.154500 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:24.154601 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 13:09:24.154659 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 13:09:24.155233 3385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:24.156661 3385 log.cc:824] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892: Log is configured to *not* fsync() on all Append() calls
I20260808 13:09:24.159502 3385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892: No bootstrap required, opened a new log
I20260808 13:09:24.167161 3385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7218ff0fd7ad4f9a9dd1a27d1f381892" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 35115 } }
I20260808 13:09:24.167569 3385 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:09:24.167681 3385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7218ff0fd7ad4f9a9dd1a27d1f381892, State: Initialized, Role: FOLLOWER
I20260808 13:09:24.168365 3385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7218ff0fd7ad4f9a9dd1a27d1f381892" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 35115 } }
I20260808 13:09:24.168604 3385 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:09:24.168789 3385 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:09:24.169003 3385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:24.171329 3385 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7218ff0fd7ad4f9a9dd1a27d1f381892" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 35115 } }
I20260808 13:09:24.171849 3385 leader_election.cc:302] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7218ff0fd7ad4f9a9dd1a27d1f381892; no voters:
I20260808 13:09:24.172571 3385 leader_election.cc:288] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:09:24.172787 3402 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:09:24.173570 3402 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892 [term 1 LEADER]: Becoming Leader. State: Replica: 7218ff0fd7ad4f9a9dd1a27d1f381892, State: Running, Role: LEADER
I20260808 13:09:24.174233 3402 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7218ff0fd7ad4f9a9dd1a27d1f381892" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 35115 } }
I20260808 13:09:24.175101 3385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:09:24.177474 3404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7218ff0fd7ad4f9a9dd1a27d1f381892. Latest consensus state: current_term: 1 leader_uuid: "7218ff0fd7ad4f9a9dd1a27d1f381892" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7218ff0fd7ad4f9a9dd1a27d1f381892" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 35115 } } }
I20260808 13:09:24.178007 3404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:24.178306 3403 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7218ff0fd7ad4f9a9dd1a27d1f381892" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7218ff0fd7ad4f9a9dd1a27d1f381892" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 35115 } } }
I20260808 13:09:24.178663 3403 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:24.182143 3412 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:09:24.187315 3412 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:09:24.196442 3412 catalog_manager.cc:1347] Generated new cluster ID: b1da9557d9044b019f7cdbf27287a9ba
I20260808 13:09:24.196589 3412 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:09:24.223994 3412 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:09:24.224550 3412 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:09:24.230723 3412 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7218ff0fd7ad4f9a9dd1a27d1f381892: Generated new TSK 0
I20260808 13:09:24.231575 3412 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:09:24.244103 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 3290
W20260808 13:09:24.245200 3378 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:35517
W20260808 13:09:24.245870 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 3290: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:09:24.246038 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 3290
2026-08-08T13:09:24Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2290 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:09:24 dist-test-slave-t3q3 krb5kdc[3434](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:09:24 dist-test-slave-t3q3 krb5kdc[3434](info): set up 2 sockets
Aug 08 13:09:24 dist-test-slave-t3q3 krb5kdc[3434](info): commencing operation
krb5kdc: starting...
W20260808 13:09:25.616614 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.326s user 0.001s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:09:25 dist-test-slave-t3q3 krb5kdc[3434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:09:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:09:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:09:25.967340 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:38965
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:43425
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:38965
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:26.318989 3451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:26.319350 3451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:26.319530 3451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:26.326776 3451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:09:26.326943 3451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:26.327071 3451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:26.327185 3451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:09:26.327267 3451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:09:26.337935 3451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:43425
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:38965
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:38965
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:26.339401 3451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:26.340796 3451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:26.353268 3459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:26.353331 3461 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:26.353358 3464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:26.354308 3451 server_base.cc:1034] running on GCE node
I20260808 13:09:26.355309 3451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:26.357033 3451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:26.358307 3451 hybrid_clock.cc:648] HybridClock initialized: now 1786194566358252 us; error 39 us; skew 500 ppm
I20260808 13:09:26.358736 3451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:26 dist-test-slave-t3q3 krb5kdc[3434](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194566, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:26.362524 3451 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:09:26.364286 3451 webserver.cc:460] Webserver started at http://127.1.124.190:35669/ using document root <none> and password file <none>
I20260808 13:09:26.365008 3451 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:26.365178 3451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:26.365509 3451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:26.368721 3451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "8d6394934cc74a2f8eff3f8b870ba5af"
format_stamp: "Formatted at 2026-08-08 13:09:26 on dist-test-slave-t3q3"
server_key: "980735481ab28fd77d3eeb92fbbf6d49"
server_key_iv: "bc1c47241290c4c746b03c6b1aff7729"
server_key_version: "encryptionkey@0"
I20260808 13:09:26.369552 3451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "8d6394934cc74a2f8eff3f8b870ba5af"
format_stamp: "Formatted at 2026-08-08 13:09:26 on dist-test-slave-t3q3"
server_key: "980735481ab28fd77d3eeb92fbbf6d49"
server_key_iv: "bc1c47241290c4c746b03c6b1aff7729"
server_key_version: "encryptionkey@0"
I20260808 13:09:26.375032 3451 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 13:09:26.378620 3474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:26.379946 3451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:09:26.380198 3451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "8d6394934cc74a2f8eff3f8b870ba5af"
format_stamp: "Formatted at 2026-08-08 13:09:26 on dist-test-slave-t3q3"
server_key: "980735481ab28fd77d3eeb92fbbf6d49"
server_key_iv: "bc1c47241290c4c746b03c6b1aff7729"
server_key_version: "encryptionkey@0"
I20260808 13:09:26.380410 3451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:26.405225 3451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:26.410321 3451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:26.410772 3451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:26.431105 3451 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:38965
I20260808 13:09:26.431129 3553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:38965 every 8 connection(s)
I20260808 13:09:26.432762 3451 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:09:26.436641 3557 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:26.442265 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 3451
I20260808 13:09:26.442502 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:09:26.442988 1522 external_mini_cluster.cc:1428] Setting key b22d1f623098a5fd5714c1b8d1954763
I20260808 13:09:26.449016 3557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af: Bootstrap starting.
I20260808 13:09:26.452950 3557 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:26.454030 3557 log.cc:824] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af: Log is configured to *not* fsync() on all Append() calls
I20260808 13:09:26.457288 3557 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af: No bootstrap required, opened a new log
Aug 08 13:09:26 dist-test-slave-t3q3 krb5kdc[3434](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194565, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:09:26.465592 3557 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6394934cc74a2f8eff3f8b870ba5af" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38965 } }
I20260808 13:09:26.466010 3557 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:09:26.466174 3557 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d6394934cc74a2f8eff3f8b870ba5af, State: Initialized, Role: FOLLOWER
I20260808 13:09:26.466909 3557 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6394934cc74a2f8eff3f8b870ba5af" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38965 } }
I20260808 13:09:26.467200 3557 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:09:26.467370 3557 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:09:26.467578 3557 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:26.467576 3561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:26.445011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59759 (local address 127.1.124.190:38965)
0808 13:09:26.445957 (+ 946us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:26.445978 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:26.446043 (+ 65us) server_negotiation.cc:407] Connection header received
0808 13:09:26.448454 (+ 2411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:26.448483 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:26.449014 (+ 531us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:26.449479 (+ 465us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:26.450827 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:26.452893 (+ 2066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:26.454345 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:26.455250 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:26.455471 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:26.459801 (+ 4330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:26.459809 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:26.459825 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:26.459865 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:26.463139 (+ 3274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:26.463900 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:26.463907 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:26.463914 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:26.464160 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:26.464583 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:26.464590 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:26.464596 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:26.465166 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:26.465476 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:26.465987 (+ 511us) server_negotiation.cc:299] Negotiation successful
0808 13:09:26.466431 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":694,"thread_start_us":457,"threads_started":1}
I20260808 13:09:26.471156 3557 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6394934cc74a2f8eff3f8b870ba5af" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38965 } }
I20260808 13:09:26.471781 3557 leader_election.cc:302] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d6394934cc74a2f8eff3f8b870ba5af; no voters:
I20260808 13:09:26.472157 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:09:26.472532 3557 leader_election.cc:288] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:09:26.472671 3565 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:09:26.474570 3557 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:09:26.481976 3565 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af [term 1 LEADER]: Becoming Leader. State: Replica: 8d6394934cc74a2f8eff3f8b870ba5af, State: Running, Role: LEADER
I20260808 13:09:26.482842 3565 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6394934cc74a2f8eff3f8b870ba5af" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38965 } }
I20260808 13:09:26.486042 3566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d6394934cc74a2f8eff3f8b870ba5af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6394934cc74a2f8eff3f8b870ba5af" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38965 } } }
I20260808 13:09:26.486465 3566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:26.487057 3567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d6394934cc74a2f8eff3f8b870ba5af. Latest consensus state: current_term: 1 leader_uuid: "8d6394934cc74a2f8eff3f8b870ba5af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d6394934cc74a2f8eff3f8b870ba5af" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 38965 } } }
I20260808 13:09:26.488116 3567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:26.487561 3588 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:09:26.503441 3588 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:09:26.504889 3561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:26.482528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39733 (local address 127.1.124.190:38965)
0808 13:09:26.482775 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:26.482784 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:26.489736 (+ 6952us) server_negotiation.cc:407] Connection header received
0808 13:09:26.490109 (+ 373us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:26.490117 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:26.490229 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:26.490321 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:26.492946 (+ 2625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:26.493726 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:26.495031 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:26.495487 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:26.495670 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:26.498175 (+ 2505us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:26.498180 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:26.498183 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:26.498216 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:26.501934 (+ 3718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:26.502648 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:26.502654 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:26.502657 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:26.502792 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:26.503231 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:26.503241 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:26.503249 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:26.503618 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:26.503882 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:26.504315 (+ 433us) server_negotiation.cc:299] Negotiation successful
0808 13:09:26.504549 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 13:09:26.513347 3588 catalog_manager.cc:1347] Generated new cluster ID: ce93a54a1e194273bb7105ef5ff45cde
I20260808 13:09:26.513466 3588 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:09:26.538779 3588 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:09:26.540689 3588 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:09:26.554334 3588 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d6394934cc74a2f8eff3f8b870ba5af: Generated new TSK 0
I20260808 13:09:26.555177 3588 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:09:26.561877 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 3451
I20260808 13:09:26.562932 3548 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:09:26.655834 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 3451
2026-08-08T13:09:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2411 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:09:26 dist-test-slave-t3q3 krb5kdc[3612](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:09:26 dist-test-slave-t3q3 krb5kdc[3612](info): set up 2 sockets
Aug 08 13:09:26 dist-test-slave-t3q3 krb5kdc[3612](info): commencing operation
krb5kdc: starting...
W20260808 13:09:28.058619 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.356s user 0.007s sys 0.047s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:09:28 dist-test-slave-t3q3 krb5kdc[3612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194568, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:09:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:09:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:09:28.411943 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:41645
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:34807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:41645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:28.768950 3627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:28.769284 3627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:28.769454 3627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:28.776770 3627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:09:28.776906 3627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:28.777019 3627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:28.777163 3627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:09:28.777240 3627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:09:28.787194 3627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:34807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:41645
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:41645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:28.788578 3627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:28.789904 3627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:28.801944 3635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:28.802009 3637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:28.802762 3640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:28.803187 3627 server_base.cc:1034] running on GCE node
I20260808 13:09:28.804024 3627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:28.805637 3627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:28.806855 3627 hybrid_clock.cc:648] HybridClock initialized: now 1786194568806809 us; error 36 us; skew 500 ppm
I20260808 13:09:28.807276 3627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:28 dist-test-slave-t3q3 krb5kdc[3612](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194568, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:28.810480 3627 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:09:28.811985 3627 webserver.cc:460] Webserver started at http://127.1.124.190:33655/ using document root <none> and password file <none>
I20260808 13:09:28.812691 3627 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:28.812839 3627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:28.813170 3627 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:28.816275 3627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "3667559bfb0143a381aceb36d8260e75"
format_stamp: "Formatted at 2026-08-08 13:09:28 on dist-test-slave-t3q3"
server_key: "96e67f3ebb6430c12bc271cb0f8b5f7b"
server_key_iv: "33b6fb3aaa344f8e7bb6b76a9fae1fd3"
server_key_version: "encryptionkey@0"
I20260808 13:09:28.817056 3627 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "3667559bfb0143a381aceb36d8260e75"
format_stamp: "Formatted at 2026-08-08 13:09:28 on dist-test-slave-t3q3"
server_key: "96e67f3ebb6430c12bc271cb0f8b5f7b"
server_key_iv: "33b6fb3aaa344f8e7bb6b76a9fae1fd3"
server_key_version: "encryptionkey@0"
I20260808 13:09:28.822348 3627 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:09:28.826054 3649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:28.827495 3627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:09:28.827705 3627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "3667559bfb0143a381aceb36d8260e75"
format_stamp: "Formatted at 2026-08-08 13:09:28 on dist-test-slave-t3q3"
server_key: "96e67f3ebb6430c12bc271cb0f8b5f7b"
server_key_iv: "33b6fb3aaa344f8e7bb6b76a9fae1fd3"
server_key_version: "encryptionkey@0"
I20260808 13:09:28.827936 3627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:28.850905 3627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:28.855232 3627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:28.855564 3627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:28.875108 3627 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:41645
I20260808 13:09:28.875159 3728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:41645 every 8 connection(s)
I20260808 13:09:28.876544 3627 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:09:28.880373 3729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:28.886759 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 3627
I20260808 13:09:28.886961 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:09:28.887547 1522 external_mini_cluster.cc:1428] Setting key bccc5514914e1aeb01e85be125a17551
I20260808 13:09:28.891422 3729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75: Bootstrap starting.
I20260808 13:09:28.895000 3729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:28.896250 3729 log.cc:824] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75: Log is configured to *not* fsync() on all Append() calls
I20260808 13:09:28.899492 3729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75: No bootstrap required, opened a new log
I20260808 13:09:28.902053 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:28.889530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34375 (local address 127.1.124.190:41645)
0808 13:09:28.890333 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:28.890344 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:28.890384 (+ 40us) server_negotiation.cc:407] Connection header received
0808 13:09:28.892413 (+ 2029us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:28.892445 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:28.893074 (+ 629us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:28.893547 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:28.894156 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:28.895533 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:28.897338 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:28.898083 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:28.898237 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:28.899508 (+ 1271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:28.899518 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:28.899533 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:09:28.900215 (+ 682us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:28.900293 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:09:28.900322 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:28.900409 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:28.900654 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 13:09:28.900928 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":369,"threads_started":1}
I20260808 13:09:28.906019 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:09:28.909005 3729 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3667559bfb0143a381aceb36d8260e75" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41645 } }
I20260808 13:09:28.909557 3729 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:09:28.909765 3729 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3667559bfb0143a381aceb36d8260e75, State: Initialized, Role: FOLLOWER
I20260808 13:09:28.910522 3729 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3667559bfb0143a381aceb36d8260e75" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41645 } }
I20260808 13:09:28.910820 3729 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:09:28.910974 3729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:09:28.911155 3729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:28.914553 3729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3667559bfb0143a381aceb36d8260e75" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41645 } }
I20260808 13:09:28.915086 3729 leader_election.cc:302] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3667559bfb0143a381aceb36d8260e75; no voters:
I20260808 13:09:28.915793 3729 leader_election.cc:288] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:09:28.915922 3745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:09:28.917793 3745 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75 [term 1 LEADER]: Becoming Leader. State: Replica: 3667559bfb0143a381aceb36d8260e75, State: Running, Role: LEADER
I20260808 13:09:28.918552 3729 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:09:28.918445 3745 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3667559bfb0143a381aceb36d8260e75" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41645 } }
I20260808 13:09:28.919029 3732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:28.911770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57785 (local address 127.1.124.190:41645)
0808 13:09:28.911998 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:28.912004 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:28.912427 (+ 423us) server_negotiation.cc:407] Connection header received
0808 13:09:28.912601 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:28.912606 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:28.912694 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:28.912817 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:28.913282 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:28.914365 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:28.915844 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:28.916732 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:28.916857 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:28.918228 (+ 1371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:28.918237 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:28.918247 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:09:28.918271 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:28.918317 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:09:28.918340 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:28.918420 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:28.918602 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 13:09:28.918700 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 13:09:28.926378 3747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3667559bfb0143a381aceb36d8260e75. Latest consensus state: current_term: 1 leader_uuid: "3667559bfb0143a381aceb36d8260e75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3667559bfb0143a381aceb36d8260e75" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41645 } } }
I20260808 13:09:28.926895 3747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:28.926349 3746 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3667559bfb0143a381aceb36d8260e75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3667559bfb0143a381aceb36d8260e75" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 41645 } } }
I20260808 13:09:28.927728 3746 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75 [sys.catalog]: This master's current role is: LEADER
W20260808 13:09:28.932000 3763 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:09:28.932158 3763 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:09:28.932304 3765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:09:28.934135 3765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:09:28.942507 3765 catalog_manager.cc:1347] Generated new cluster ID: fe6eda47978040c182791eb75c9dd380
I20260808 13:09:28.942631 3765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:09:28.959808 3765 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:09:28.961306 3765 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:09:28.967308 3765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3667559bfb0143a381aceb36d8260e75: Generated new TSK 0
I20260808 13:09:28.968408 3765 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:09:28.983026 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 3627
W20260808 13:09:28.984244 3722 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:34375
W20260808 13:09:28.985065 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 3627: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:09:28.985245 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 3627
2026-08-08T13:09:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2332 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:09:29 dist-test-slave-t3q3 krb5kdc[3774](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:09:29 dist-test-slave-t3q3 krb5kdc[3774](info): set up 2 sockets
Aug 08 13:09:29 dist-test-slave-t3q3 krb5kdc[3774](info): commencing operation
krb5kdc: starting...
W20260808 13:09:30.346763 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.312s user 0.001s sys 0.051s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:09:30 dist-test-slave-t3q3 krb5kdc[3774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194570, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:09:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:09:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:09:30.697762 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:33311
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:43795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:33311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:31.067714 3789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:31.068128 3789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:31.068416 3789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:31.076363 3789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:09:31.076578 3789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:31.076699 3789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:31.076821 3789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:09:31.076903 3789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:09:31.087045 3789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:43795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:33311
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:33311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:31.088542 3789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:31.089908 3789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:31.102538 3801 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:31.102506 3802 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:31.102645 3804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:31.103609 3789 server_base.cc:1034] running on GCE node
I20260808 13:09:31.104662 3789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:31.106208 3789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:31.107445 3789 hybrid_clock.cc:648] HybridClock initialized: now 1786194571107388 us; error 41 us; skew 500 ppm
I20260808 13:09:31.107874 3789 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:31 dist-test-slave-t3q3 krb5kdc[3774](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194571, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:31.111362 3789 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:09:31.113126 3789 webserver.cc:460] Webserver started at http://127.1.124.190:40745/ using document root <none> and password file <none>
I20260808 13:09:31.113840 3789 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:31.113974 3789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:31.114305 3789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:31.117470 3789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "150deed1b34f4cd49ef0bdd72c1c0dc9"
format_stamp: "Formatted at 2026-08-08 13:09:31 on dist-test-slave-t3q3"
server_key: "a4020276efe37c5aaaa6e398f01eb7fe"
server_key_iv: "be58b496cf95155e4ea17d60b0047b1b"
server_key_version: "encryptionkey@0"
I20260808 13:09:31.118258 3789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "150deed1b34f4cd49ef0bdd72c1c0dc9"
format_stamp: "Formatted at 2026-08-08 13:09:31 on dist-test-slave-t3q3"
server_key: "a4020276efe37c5aaaa6e398f01eb7fe"
server_key_iv: "be58b496cf95155e4ea17d60b0047b1b"
server_key_version: "encryptionkey@0"
I20260808 13:09:31.123677 3789 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260808 13:09:31.127360 3812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:31.128852 3789 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:09:31.129061 3789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "150deed1b34f4cd49ef0bdd72c1c0dc9"
format_stamp: "Formatted at 2026-08-08 13:09:31 on dist-test-slave-t3q3"
server_key: "a4020276efe37c5aaaa6e398f01eb7fe"
server_key_iv: "be58b496cf95155e4ea17d60b0047b1b"
server_key_version: "encryptionkey@0"
I20260808 13:09:31.129276 3789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:31.135371 3789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:31.139869 3789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:31.140311 3789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:31.159641 3789 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:33311
I20260808 13:09:31.159665 3891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:33311 every 8 connection(s)
I20260808 13:09:31.161177 3789 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:09:31.163010 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 3789
I20260808 13:09:31.163205 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:09:31.163770 1522 external_mini_cluster.cc:1428] Setting key 8e28285cc5c95670808cc9b2da349dd4
I20260808 13:09:31.165678 3893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:31.171685 3898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:31.165672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56575 (local address 127.1.124.190:33311)
0808 13:09:31.166481 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:31.166494 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:31.166535 (+ 41us) server_negotiation.cc:407] Connection header received
0808 13:09:31.168230 (+ 1695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:31.168251 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:31.168597 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:31.169005 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:31.170058 (+ 1053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:31.170064 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:31.170076 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:09:31.170089 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:31.170164 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:09:31.170184 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:31.170434 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:31.170594 (+ 160us) server_negotiation.cc:299] Negotiation successful
0808 13:09:31.170773 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":362,"threads_started":1}
I20260808 13:09:31.175379 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:09:31.177513 3893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9: Bootstrap starting.
I20260808 13:09:31.180883 3893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:31.182210 3893 log.cc:824] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9: Log is configured to *not* fsync() on all Append() calls
I20260808 13:09:31.182586 3898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:31.179808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49447 (local address 127.1.124.190:33311)
0808 13:09:31.180003 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:31.180011 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:31.180570 (+ 559us) server_negotiation.cc:407] Connection header received
0808 13:09:31.180701 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:31.180706 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:31.180782 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:31.180908 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:31.181927 (+ 1019us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:31.181935 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:31.181942 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 13:09:31.181964 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:31.182019 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 13:09:31.182042 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:31.182162 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:31.182257 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 13:09:31.182316 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 13:09:31.185452 3893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9: No bootstrap required, opened a new log
I20260808 13:09:31.193087 3893 raft_consensus.cc:348] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150deed1b34f4cd49ef0bdd72c1c0dc9" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 33311 } }
I20260808 13:09:31.193444 3893 raft_consensus.cc:374] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 13:09:31.193545 3893 raft_consensus.cc:729] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 150deed1b34f4cd49ef0bdd72c1c0dc9, State: Initialized, Role: FOLLOWER
I20260808 13:09:31.194229 3893 consensus_queue.cc:260] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150deed1b34f4cd49ef0bdd72c1c0dc9" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 33311 } }
I20260808 13:09:31.194456 3893 raft_consensus.cc:388] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 13:09:31.194619 3893 raft_consensus.cc:482] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 13:09:31.194799 3893 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:31.197036 3893 raft_consensus.cc:504] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150deed1b34f4cd49ef0bdd72c1c0dc9" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 33311 } }
I20260808 13:09:31.197530 3893 leader_election.cc:302] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 150deed1b34f4cd49ef0bdd72c1c0dc9; no voters:
I20260808 13:09:31.198172 3893 leader_election.cc:288] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 13:09:31.198323 3911 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:09:31.199010 3911 raft_consensus.cc:686] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9 [term 1 LEADER]: Becoming Leader. State: Replica: 150deed1b34f4cd49ef0bdd72c1c0dc9, State: Running, Role: LEADER
I20260808 13:09:31.199608 3911 consensus_queue.cc:237] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150deed1b34f4cd49ef0bdd72c1c0dc9" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 33311 } }
I20260808 13:09:31.200325 3893 sys_catalog.cc:564] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:09:31.202550 3912 sys_catalog.cc:455] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "150deed1b34f4cd49ef0bdd72c1c0dc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150deed1b34f4cd49ef0bdd72c1c0dc9" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 33311 } } }
I20260808 13:09:31.202984 3912 sys_catalog.cc:458] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:31.202983 3913 sys_catalog.cc:455] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 150deed1b34f4cd49ef0bdd72c1c0dc9. Latest consensus state: current_term: 1 leader_uuid: "150deed1b34f4cd49ef0bdd72c1c0dc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "150deed1b34f4cd49ef0bdd72c1c0dc9" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 33311 } } }
I20260808 13:09:31.203831 3913 sys_catalog.cc:458] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:31.205986 3922 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:09:31.211630 3922 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 13:09:31.220211 3922 catalog_manager.cc:1347] Generated new cluster ID: ccb67f202d6e421291a9767c5bfff243
I20260808 13:09:31.220320 3922 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:09:31.254762 3922 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:09:31.255209 3922 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:09:31.264024 3922 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 150deed1b34f4cd49ef0bdd72c1c0dc9: Generated new TSK 0
I20260808 13:09:31.265071 3922 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:09:31.274137 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 3789
W20260808 13:09:31.275272 3885 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:56575
W20260808 13:09:31.275996 1522 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 3789: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260808 13:09:31.276207 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 3789
2026-08-08T13:09:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2288 ms)
[----------] 18 tests from AuthTokenIssuingTest (41893 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:09:31 dist-test-slave-t3q3 krb5kdc[3945](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:09:31 dist-test-slave-t3q3 krb5kdc[3945](info): set up 2 sockets
Aug 08 13:09:31 dist-test-slave-t3q3 krb5kdc[3945](info): commencing operation
krb5kdc: starting...
W20260808 13:09:32.625581 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:09:32 dist-test-slave-t3q3 krb5kdc[3945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:09:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:09:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:09:32.991883 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:44807
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:41181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:44807,127.1.124.189:43019,127.1.124.188:33689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:33.344702 3965 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 13:09:33.345175 3965 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:33.345314 3965 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:33.345474 3965 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:33.352738 3965 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:09:33.352885 3965 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:33.353013 3965 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:33.353107 3965 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:09:33.353201 3965 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:09:33.363178 3965 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:41181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:44807,127.1.124.189:43019,127.1.124.188:33689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:44807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:33.364579 3965 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:33.366009 3965 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:33.378283 3973 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:33.378407 3972 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:33.378365 3978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:33.379325 3965 server_base.cc:1034] running on GCE node
I20260808 13:09:33.380141 3965 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:33.381512 3965 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:33.382772 3965 hybrid_clock.cc:648] HybridClock initialized: now 1786194573382707 us; error 35 us; skew 500 ppm
I20260808 13:09:33.383116 3965 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:33 dist-test-slave-t3q3 krb5kdc[3945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194573, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:33.390194 3965 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:09:33.391782 3965 webserver.cc:460] Webserver started at http://127.1.124.190:39429/ using document root <none> and password file <none>
I20260808 13:09:33.392560 3965 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:33.392695 3965 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:33.393050 3965 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:33.396127 3965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "85505a49c3e64a1a8aed972e438643ad"
format_stamp: "Formatted at 2026-08-08 13:09:33 on dist-test-slave-t3q3"
server_key: "7a1b5c8c24b541aa265d451484d60fdd"
server_key_iv: "d35edd070f9b0053e18d501f22419b06"
server_key_version: "encryptionkey@0"
I20260808 13:09:33.396940 3965 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "85505a49c3e64a1a8aed972e438643ad"
format_stamp: "Formatted at 2026-08-08 13:09:33 on dist-test-slave-t3q3"
server_key: "7a1b5c8c24b541aa265d451484d60fdd"
server_key_iv: "d35edd070f9b0053e18d501f22419b06"
server_key_version: "encryptionkey@0"
I20260808 13:09:33.402331 3965 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:09:33.406046 3986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:33.407550 3965 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:09:33.407759 3965 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "85505a49c3e64a1a8aed972e438643ad"
format_stamp: "Formatted at 2026-08-08 13:09:33 on dist-test-slave-t3q3"
server_key: "7a1b5c8c24b541aa265d451484d60fdd"
server_key_iv: "d35edd070f9b0053e18d501f22419b06"
server_key_version: "encryptionkey@0"
I20260808 13:09:33.408001 3965 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:33.427271 3965 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:33.431623 3965 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:33.431931 3965 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:33.451745 3965 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:44807
I20260808 13:09:33.451778 4063 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:44807 every 8 connection(s)
I20260808 13:09:33.453228 3965 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:09:33.457109 4064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:33.457248 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 3965
I20260808 13:09:33.457484 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:09:33.458168 1522 external_mini_cluster.cc:1428] Setting key 503176a60e9f6b800c776f3eaefc25f7
I20260808 13:09:33.463776 4064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44807 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:09:33 dist-test-slave-t3q3 krb5kdc[3945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194573, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:09:33.489593 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:33.467016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:44807 (local address 127.0.0.1:49514)
0808 13:09:33.467935 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:33.467988 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:33.470830 (+ 2842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:33.471673 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:33.471683 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:33.472568 (+ 885us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:33.472909 (+ 341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:33.472926 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:33.474832 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:33.474836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:33.475892 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:33.475904 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:33.477260 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:33.477265 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:33.477386 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:33.478111 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:33.478160 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:33.480584 (+ 2424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:33.485089 (+ 4505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:33.485098 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:33.485107 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:33.485478 (+ 371us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:33.485982 (+ 504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:33.485985 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:33.485988 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:33.486169 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:33.487130 (+ 961us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:33.487140 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:33.487318 (+ 178us) client_negotiation.cc:769] Sending connection context
0808 13:09:33.487578 (+ 260us) client_negotiation.cc:240] Negotiation successful
0808 13:09:33.488123 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"mutex_wait_us":3,"thread_start_us":373,"threads_started":1}
I20260808 13:09:33.489631 4069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:33.467016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49514 (local address 127.1.124.190:44807)
0808 13:09:33.468926 (+ 1910us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:33.468932 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:33.468974 (+ 42us) server_negotiation.cc:407] Connection header received
0808 13:09:33.471224 (+ 2250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:33.471229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:33.471436 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:33.471597 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:33.473179 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:33.474685 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:33.476042 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:33.477038 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:33.477296 (+ 258us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:33.480824 (+ 3528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:33.480837 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:33.480856 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:33.480904 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:33.484950 (+ 4046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:33.485637 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:33.485649 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:33.485656 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:33.485859 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:33.486294 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:33.486302 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:33.486308 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:33.486981 (+ 673us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:33.487242 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:33.487821 (+ 579us) server_negotiation.cc:299] Negotiation successful
0808 13:09:33.488321 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":220,"server-negotiator.queue_time_us":1726,"spinlock_wait_cycles":52096,"thread_start_us":1481,"threads_started":1}
I20260808 13:09:33.492588 4064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:09:33.493733 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:33.493228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:43019 (local address 127.0.0.1:44916)
0808 13:09:33.493436 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:33.493574 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:43019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260808 13:09:33.494007 3987 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.124.189:43019: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.1.124.189@KRBTEST.COM; defaulting to no policy
I20260808 13:09:33.495683 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:33.495356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:43019 (local address 127.0.0.1:44920)
0808 13:09:33.495497 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:33.495563 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:43019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 13:09:33.495879 4064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.189:43019: Network error: Client connection negotiation failed: client connection to 127.1.124.189:43019: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.189.keytab.
Entry for principal kudu/127.1.124.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.189.keytab.
I20260808 13:09:33.533694 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.189:43019
--webserver_interface=127.1.124.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.189.keytab
--principal=kudu/127.1.124.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:41181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:44807,127.1.124.189:43019,127.1.124.188:33689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:09:33.545239 4064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } attempt: 1
I20260808 13:09:33.547168 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:33.546519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:43019 (local address 127.0.0.1:44936)
0808 13:09:33.546804 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:33.546957 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:43019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 13:09:33.548349 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:33.547856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:43019 (local address 127.0.0.1:44944)
0808 13:09:33.548046 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:33.548174 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:43019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260808 13:09:33.548553 4064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.189:43019: Network error: Client connection negotiation failed: client connection to 127.1.124.189:43019: connect: Connection refused (error 111)
I20260808 13:09:33.616832 4064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } attempt: 2
I20260808 13:09:33.618250 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:33.617674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:43019 (local address 127.0.0.1:44948)
0808 13:09:33.617943 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:33.618060 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:43019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 13:09:33.619522 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:33.619030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:43019 (local address 127.0.0.1:44954)
0808 13:09:33.619248 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:33.619361 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:43019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260808 13:09:33.619768 4064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.189:43019: Network error: Client connection negotiation failed: client connection to 127.1.124.189:43019: connect: Connection refused (error 111)
I20260808 13:09:33.711165 4064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } attempt: 3
I20260808 13:09:33.712530 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:33.711942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:43019 (local address 127.0.0.1:44956)
0808 13:09:33.712246 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:33.712362 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:43019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153}
I20260808 13:09:33.713686 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:33.713262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:43019 (local address 127.0.0.1:44970)
0808 13:09:33.713431 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:33.713517 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:43019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260808 13:09:33.713910 4064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.189:43019: Network error: Client connection negotiation failed: client connection to 127.1.124.189:43019: connect: Connection refused (error 111)
I20260808 13:09:33.857200 4064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } attempt: 4
I20260808 13:09:33.858618 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:33.857946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:43019 (local address 127.0.0.1:44976)
0808 13:09:33.858330 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:33.858445 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:43019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219}
I20260808 13:09:33.859786 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:33.859333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:43019 (local address 127.0.0.1:44988)
0808 13:09:33.859503 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:33.859614 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:43019: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260808 13:09:33.860074 4064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.189:43019: Network error: Client connection negotiation failed: client connection to 127.1.124.189:43019: connect: Connection refused (error 111)
W20260808 13:09:33.885857 4076 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 13:09:33.886178 4076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:33.886250 4076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:33.886332 4076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:33.893795 4076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:09:33.893893 4076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:33.893950 4076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:33.894001 4076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:09:33.894057 4076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:09:33.904000 4076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:41181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:44807,127.1.124.189:43019,127.1.124.188:33689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.189.keytab
--principal=kudu/127.1.124.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.189:43019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.124.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:33.905546 4076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:33.906962 4076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:33.920727 4084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:33.920925 4086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:33.920881 4083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:33.922708 4076 server_base.cc:1034] running on GCE node
I20260808 13:09:33.923741 4076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:33.925499 4076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:33.926810 4076 hybrid_clock.cc:648] HybridClock initialized: now 1786194573926729 us; error 57 us; skew 500 ppm
I20260808 13:09:33.927276 4076 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:33 dist-test-slave-t3q3 krb5kdc[3945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194573, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:33.930716 4076 init.cc:377] Logged in from keytab as kudu/127.1.124.189@KRBTEST.COM (short username kudu)
I20260808 13:09:33.932338 4076 webserver.cc:460] Webserver started at http://127.1.124.189:36889/ using document root <none> and password file <none>
I20260808 13:09:33.933054 4076 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:33.933148 4076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:33.933431 4076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:33.936765 4076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-1/data/instance:
uuid: "e72328649fed4f7da247d3b895cb5ccc"
format_stamp: "Formatted at 2026-08-08 13:09:33 on dist-test-slave-t3q3"
server_key: "f2df8fa57b174e6fd52438c6faa711c6"
server_key_iv: "b60d7900fd272cd44caa50bad03fdfdf"
server_key_version: "encryptionkey@0"
I20260808 13:09:33.937575 4076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-1/wal/instance:
uuid: "e72328649fed4f7da247d3b895cb5ccc"
format_stamp: "Formatted at 2026-08-08 13:09:33 on dist-test-slave-t3q3"
server_key: "f2df8fa57b174e6fd52438c6faa711c6"
server_key_iv: "b60d7900fd272cd44caa50bad03fdfdf"
server_key_version: "encryptionkey@0"
I20260808 13:09:33.943109 4076 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260808 13:09:33.946929 4096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:33.948446 4076 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 13:09:33.948655 4076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-1/wal
uuid: "e72328649fed4f7da247d3b895cb5ccc"
format_stamp: "Formatted at 2026-08-08 13:09:33 on dist-test-slave-t3q3"
server_key: "f2df8fa57b174e6fd52438c6faa711c6"
server_key_iv: "b60d7900fd272cd44caa50bad03fdfdf"
server_key_version: "encryptionkey@0"
I20260808 13:09:33.948911 4076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:33.971148 4076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:33.975544 4076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:33.975845 4076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:33.995643 4076 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.189:43019
I20260808 13:09:33.995666 4168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.189:43019 every 8 connection(s)
I20260808 13:09:33.997092 4076 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-1/data/info.pb
I20260808 13:09:33.999275 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 4076
I20260808 13:09:33.999476 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-1/wal/instance
I20260808 13:09:34.000087 1522 external_mini_cluster.cc:1428] Setting key d8f5a58f513d6445ff0e12ecd08d3bec
I20260808 13:09:34.001675 4170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:34.012300 4170 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44807 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:09:34 dist-test-slave-t3q3 krb5kdc[3945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194573, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.189@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.124.188@KRBTEST.COM; defaulting to no policy
I20260808 13:09:34.037833 4177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.015879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49530 (local address 127.1.124.190:44807)
0808 13:09:34.016487 (+ 608us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:34.016494 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:34.017513 (+ 1019us) server_negotiation.cc:407] Connection header received
0808 13:09:34.020517 (+ 3004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:34.020522 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:34.020598 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:34.020703 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:34.022982 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.023821 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.026332 (+ 2511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.026768 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.026876 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:34.031338 (+ 4462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:34.031343 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:34.031346 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:34.031384 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:34.034483 (+ 3099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.035324 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.035329 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.035331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.035455 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.035935 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.035944 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.035946 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.036299 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:34.036457 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:34.037210 (+ 753us) server_negotiation.cc:299] Negotiation successful
0808 13:09:34.037439 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":321,"threads_started":1}
I20260808 13:09:34.038939 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.016483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:44807 (local address 127.0.0.1:49530)
0808 13:09:34.017331 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.017377 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:34.020204 (+ 2827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:34.020988 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:34.021001 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:34.021808 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:34.022737 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:34.022766 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.023995 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.024003 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:34.026162 (+ 2159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:34.026180 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.027039 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.027051 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:34.027415 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:34.028217 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:34.028255 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:34.031064 (+ 2809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:34.034667 (+ 3603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:34.034677 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:34.034695 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:34.035173 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:34.035598 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:34.035603 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:34.035606 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:34.035808 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:34.036433 (+ 625us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:34.036441 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:34.036828 (+ 387us) client_negotiation.cc:769] Sending connection context
0808 13:09:34.037115 (+ 287us) client_negotiation.cc:240] Negotiation successful
0808 13:09:34.037674 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":362,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.188@KRBTEST.COM" created.
I20260808 13:09:34.041020 4170 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:09:34 dist-test-slave-t3q3 krb5kdc[3945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194573, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.189@KRBTEST.COM for kudu/127.1.124.189@KRBTEST.COM
I20260808 13:09:34.056526 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.041939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:43019 (local address 127.0.0.1:45004)
0808 13:09:34.042154 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.042173 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:34.042269 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:34.043288 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:34.043293 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:34.043732 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:34.043853 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:34.043862 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.045336 (+ 1474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.045344 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:34.046183 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:34.046194 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.047028 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.047033 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:34.047277 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:34.047899 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:34.047916 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:34.050214 (+ 2298us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:34.053802 (+ 3588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:34.053809 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:34.053814 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:34.054294 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:34.054703 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:34.054708 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:34.054711 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:34.054847 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:34.055459 (+ 612us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:34.055468 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:34.055626 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 13:09:34.055772 (+ 146us) client_negotiation.cc:240] Negotiation successful
0808 13:09:34.056089 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":44}
I20260808 13:09:34.056586 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.042219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45004 (local address 127.1.124.189:43019)
0808 13:09:34.042871 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:34.042877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:34.042902 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:09:34.042949 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:34.042955 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:34.043139 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:34.043306 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:34.044031 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.045197 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.046333 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.046879 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.047020 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:34.050446 (+ 3426us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:34.050458 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:34.050472 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:34.050501 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:34.053671 (+ 3170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.054429 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.054441 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.054444 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.054560 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.054952 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.054959 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.054962 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.055328 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:34.055551 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:34.055896 (+ 345us) server_negotiation.cc:299] Negotiation successful
0808 13:09:34.056198 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":72,"server-negotiator.queue_time_us":514,"thread_start_us":383,"threads_started":1}
I20260808 13:09:34.059353 4170 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:09:34.060725 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.060244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:33689 (local address 127.0.0.1:40118)
0808 13:09:34.060433 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.060554 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
W20260808 13:09:34.061038 4100 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
I20260808 13:09:34.062731 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.062369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:33689 (local address 127.0.0.1:40130)
0808 13:09:34.062555 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.062626 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 13:09:34.062965 4170 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:33689: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.188.keytab.
Entry for principal kudu/127.1.124.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.188.keytab.
I20260808 13:09:34.084515 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.188:33689
--webserver_interface=127.1.124.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.188.keytab
--principal=kudu/127.1.124.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:41181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:44807,127.1.124.189:43019,127.1.124.188:33689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:09:34.112262 4170 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } attempt: 1
I20260808 13:09:34.113715 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.113025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:33689 (local address 127.0.0.1:40140)
0808 13:09:34.113284 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.113456 (+ 172us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 13:09:34.115111 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.114612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:33689 (local address 127.0.0.1:40142)
0808 13:09:34.114821 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.114940 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 13:09:34.115382 4170 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:33689: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
I20260808 13:09:34.159400 4064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } attempt: 5
Aug 08 13:09:34 dist-test-slave-t3q3 krb5kdc[3945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194573, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for kudu/127.1.124.189@KRBTEST.COM
I20260808 13:09:34.175302 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.160372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45006 (local address 127.1.124.189:43019)
0808 13:09:34.160569 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:34.160584 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:34.160696 (+ 112us) server_negotiation.cc:407] Connection header received
0808 13:09:34.160955 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:34.160960 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:34.161027 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:34.161124 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:34.162148 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.163111 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.164868 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.165355 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.165461 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:34.169598 (+ 4137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:34.169603 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:34.169606 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:34.169640 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:34.172431 (+ 2791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.173316 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.173324 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.173328 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.173482 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.173959 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.173962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.173964 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.174259 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:34.174367 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:34.174857 (+ 490us) server_negotiation.cc:299] Negotiation successful
0808 13:09:34.175032 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 13:09:34.175441 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.160343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:43019 (local address 127.0.0.1:45006)
0808 13:09:34.160573 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.160597 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:34.160753 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:34.161192 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:34.161197 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:34.161740 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:34.161978 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:34.161991 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.163263 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.163269 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:34.164706 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:34.164723 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.165523 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.165528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:34.165773 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:34.166441 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:34.166466 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:34.169363 (+ 2897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:34.172576 (+ 3213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:34.172584 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:34.172590 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:34.173162 (+ 572us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:34.173611 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:34.173616 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:34.173627 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:34.173830 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:34.174384 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:34.174389 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:34.174565 (+ 176us) client_negotiation.cc:769] Sending connection context
0808 13:09:34.174730 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 13:09:34.175082 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
I20260808 13:09:34.176791 4064 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:09:34.178150 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.177623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:33689 (local address 127.0.0.1:40158)
0808 13:09:34.177835 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.177947 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 13:09:34.179203 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.178830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:33689 (local address 127.0.0.1:40160)
0808 13:09:34.178989 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.179085 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 13:09:34.179441 4064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:33689: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
I20260808 13:09:34.183615 4170 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } attempt: 2
I20260808 13:09:34.184667 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.184198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:33689 (local address 127.0.0.1:40172)
0808 13:09:34.184376 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.184495 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260808 13:09:34.185632 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.185263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:33689 (local address 127.0.0.1:40176)
0808 13:09:34.185408 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.185489 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 13:09:34.185846 4170 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:33689: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
I20260808 13:09:34.230715 4064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } attempt: 1
I20260808 13:09:34.232052 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.231469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:33689 (local address 127.0.0.1:40192)
0808 13:09:34.231724 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.231841 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260808 13:09:34.233234 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.232847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:33689 (local address 127.0.0.1:40200)
0808 13:09:34.233017 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.233107 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 13:09:34.233500 4064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:33689: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
I20260808 13:09:34.277098 4170 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } attempt: 3
I20260808 13:09:34.278403 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.277832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:33689 (local address 127.0.0.1:40204)
0808 13:09:34.278066 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.278182 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260808 13:09:34.279646 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.279126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:33689 (local address 127.0.0.1:40214)
0808 13:09:34.279373 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.279467 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141}
W20260808 13:09:34.279919 4170 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:33689: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
I20260808 13:09:34.301764 4064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } attempt: 2
I20260808 13:09:34.303274 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.302573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:33689 (local address 127.0.0.1:40230)
0808 13:09:34.302878 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.303022 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260808 13:09:34.304633 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.304126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:33689 (local address 127.0.0.1:40238)
0808 13:09:34.304341 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.304454 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 13:09:34.304917 4064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:33689: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
I20260808 13:09:34.411176 4064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } attempt: 3
I20260808 13:09:34.412541 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.411884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:33689 (local address 127.0.0.1:40248)
0808 13:09:34.412208 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.412325 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260808 13:09:34.413913 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.413389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:33689 (local address 127.0.0.1:40256)
0808 13:09:34.413653 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.413761 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260808 13:09:34.414223 4064 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:33689: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
I20260808 13:09:34.423159 4170 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } attempt: 4
I20260808 13:09:34.424376 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.423787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:33689 (local address 127.0.0.1:40264)
0808 13:09:34.424051 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.424186 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260808 13:09:34.425455 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.425087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:33689 (local address 127.0.0.1:40276)
0808 13:09:34.425247 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.425332 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260808 13:09:34.425748 4170 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:33689: Network error: Client connection negotiation failed: client connection to 127.1.124.188:33689: connect: Connection refused (error 111)
W20260808 13:09:34.448470 4186 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 13:09:34.448832 4186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:34.448923 4186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:34.449061 4186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:34.456514 4186 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:09:34.456635 4186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:34.456760 4186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:34.456847 4186 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:09:34.456956 4186 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:09:34.466871 4186 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:41181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:44807,127.1.124.189:43019,127.1.124.188:33689
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.188.keytab
--principal=kudu/127.1.124.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.188:33689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.124.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:34.468287 4186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:34.469599 4186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:34.481602 4193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:34.481774 4192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:34.481746 4196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:34.482415 4186 server_base.cc:1034] running on GCE node
I20260808 13:09:34.483265 4186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:34.484722 4186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:34.485975 4186 hybrid_clock.cc:648] HybridClock initialized: now 1786194574485919 us; error 53 us; skew 500 ppm
I20260808 13:09:34.486310 4186 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:34 dist-test-slave-t3q3 krb5kdc[3945](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194574, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:34.489554 4186 init.cc:377] Logged in from keytab as kudu/127.1.124.188@KRBTEST.COM (short username kudu)
I20260808 13:09:34.490911 4186 webserver.cc:460] Webserver started at http://127.1.124.188:39439/ using document root <none> and password file <none>
I20260808 13:09:34.491541 4186 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:34.491636 4186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:34.491907 4186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:34.494881 4186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-2/data/instance:
uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5"
format_stamp: "Formatted at 2026-08-08 13:09:34 on dist-test-slave-t3q3"
server_key: "979f4f1c9199620156a98ed65ce019fb"
server_key_iv: "03e684969ca9df551716fabd52e0f0c6"
server_key_version: "encryptionkey@0"
I20260808 13:09:34.495616 4186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-2/wal/instance:
uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5"
format_stamp: "Formatted at 2026-08-08 13:09:34 on dist-test-slave-t3q3"
server_key: "979f4f1c9199620156a98ed65ce019fb"
server_key_iv: "03e684969ca9df551716fabd52e0f0c6"
server_key_version: "encryptionkey@0"
I20260808 13:09:34.500834 4186 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260808 13:09:34.504205 4207 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:34.505599 4186 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 13:09:34.505791 4186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-2/wal
uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5"
format_stamp: "Formatted at 2026-08-08 13:09:34 on dist-test-slave-t3q3"
server_key: "979f4f1c9199620156a98ed65ce019fb"
server_key_iv: "03e684969ca9df551716fabd52e0f0c6"
server_key_version: "encryptionkey@0"
I20260808 13:09:34.506036 4186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:34.526249 4186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:34.530591 4186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:34.530900 4186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:34.551040 4186 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.188:33689
I20260808 13:09:34.551059 4286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.188:33689 every 8 connection(s)
I20260808 13:09:34.552888 4186 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-2/data/info.pb
I20260808 13:09:34.556896 4287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:34.560153 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 4186
I20260808 13:09:34.560374 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786194185613124-1522-0/minicluster-data/master-2/wal/instance
I20260808 13:09:34.560921 1522 external_mini_cluster.cc:1428] Setting key bdb56536bbb3482b7c83a4fc76ca33d1
I20260808 13:09:34.565153 4287 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44807 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:09:34 dist-test-slave-t3q3 krb5kdc[3945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.188@KRBTEST.COM
Aug 08 13:09:34 dist-test-slave-t3q3 krb5kdc[3945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194574, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.188@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:09:34.583066 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.568169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49544 (local address 127.1.124.190:44807)
0808 13:09:34.568717 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:34.568723 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:34.569250 (+ 527us) server_negotiation.cc:407] Connection header received
0808 13:09:34.569453 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:34.569459 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:34.569559 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:34.569683 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:34.570777 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.571448 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.572913 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.573341 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.573484 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:34.577207 (+ 3723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:34.577212 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:34.577215 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:34.577248 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:34.579885 (+ 2637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.580616 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.580623 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.580629 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.580782 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.581217 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.581222 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.581226 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.581502 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:34.581641 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:34.582377 (+ 736us) server_negotiation.cc:299] Negotiation successful
0808 13:09:34.582629 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":410,"thread_start_us":246,"threads_started":1}
I20260808 13:09:34.584314 4295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.568134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:44807 (local address 127.0.0.1:49544)
0808 13:09:34.569091 (+ 957us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.569142 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:34.569301 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:34.569739 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:34.569744 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:34.570365 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:34.570611 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:34.570619 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.571607 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.571618 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:34.572767 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:34.572780 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.573576 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.573584 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:34.573743 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:34.574398 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:34.574427 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:34.577015 (+ 2588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:34.580021 (+ 3006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:34.580034 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:34.580048 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:34.580469 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:34.580898 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:34.580905 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:34.580909 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:34.581090 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:34.581650 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:34.581658 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:34.581937 (+ 279us) client_negotiation.cc:769] Sending connection context
0808 13:09:34.582235 (+ 298us) client_negotiation.cc:240] Negotiation successful
0808 13:09:34.582791 (+ 556us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":719,"spinlock_wait_cycles":18304,"thread_start_us":562,"threads_started":1}
I20260808 13:09:34.584334 4293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.563083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40278 (local address 127.1.124.188:33689)
0808 13:09:34.563875 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:34.563887 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:34.563930 (+ 43us) server_negotiation.cc:407] Connection header received
0808 13:09:34.565722 (+ 1792us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:34.565745 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:34.566194 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:34.566584 (+ 390us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:34.568273 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.570157 (+ 1884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.571482 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.572262 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.572471 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:34.576935 (+ 4464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:34.576947 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:34.576962 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:34.576997 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:34.579923 (+ 2926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.580680 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.580686 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.580689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.580852 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.581261 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.581264 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.581266 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.581883 (+ 617us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:34.582221 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:34.582539 (+ 318us) server_negotiation.cc:299] Negotiation successful
0808 13:09:34.582894 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":136,"server-negotiator.queue_time_us":585,"thread_start_us":358,"threads_started":1}
I20260808 13:09:34.586176 4287 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:09:34.591466 4064 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } attempt: 4
Aug 08 13:09:34 dist-test-slave-t3q3 krb5kdc[3945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194574, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.188@KRBTEST.COM for kudu/127.1.124.189@KRBTEST.COM
Aug 08 13:09:34 dist-test-slave-t3q3 krb5kdc[3945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
Aug 08 13:09:34 dist-test-slave-t3q3 krb5kdc[3945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194573, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for kudu/127.1.124.188@KRBTEST.COM
I20260808 13:09:34.602599 4295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.587055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:43019 (local address 127.0.0.1:45018)
0808 13:09:34.587298 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.587313 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:34.587448 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:34.587774 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:34.587779 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:34.588291 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:34.588498 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:34.588509 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.589477 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.589482 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:34.590505 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:34.590550 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.591224 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.591231 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:34.591437 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:34.592518 (+ 1081us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:34.592551 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:34.595420 (+ 2869us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:34.599695 (+ 4275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:34.599702 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:34.599706 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:34.600304 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:34.600756 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:34.600762 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:34.600765 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:34.600965 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:34.601587 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:34.601592 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:34.601750 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 13:09:34.601901 (+ 151us) client_negotiation.cc:240] Negotiation successful
0808 13:09:34.602204 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160}
I20260808 13:09:34.602625 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.586993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45018 (local address 127.1.124.189:43019)
0808 13:09:34.587196 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:34.587202 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:34.587390 (+ 188us) server_negotiation.cc:407] Connection header received
0808 13:09:34.587584 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:34.587588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:34.587649 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:34.587751 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:34.588642 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.589331 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.590664 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.591075 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.591177 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:34.595646 (+ 4469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:34.595652 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:34.595659 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:34.595705 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:34.599514 (+ 3809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.600437 (+ 923us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.600444 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.600449 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.600614 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.601087 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.601094 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.601100 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.601439 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:34.601611 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:34.602014 (+ 403us) server_negotiation.cc:299] Negotiation successful
0808 13:09:34.602227 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260808 13:09:34.603982 4287 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:09:34.605144 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.587705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49552 (local address 127.1.124.190:44807)
0808 13:09:34.587869 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:34.587873 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:34.588157 (+ 284us) server_negotiation.cc:407] Connection header received
0808 13:09:34.588317 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:34.588321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:34.588382 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:34.588460 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:34.589801 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.590463 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.592053 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.592691 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.592820 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:34.597942 (+ 5122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:34.597949 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:34.597953 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:34.597992 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:34.601623 (+ 3631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.602544 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.602551 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.602556 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.602707 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.603199 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.603203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.603207 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.603588 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:34.603882 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:34.604560 (+ 678us) server_negotiation.cc:299] Negotiation successful
0808 13:09:34.604811 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 13:09:34.613083 4293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.592342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40292 (local address 127.1.124.188:33689)
0808 13:09:34.592491 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:34.592495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:34.592509 (+ 14us) server_negotiation.cc:407] Connection header received
0808 13:09:34.592545 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:34.592548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:34.592597 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:34.592708 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:34.593511 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.594564 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.596202 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.596806 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.596950 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:34.600328 (+ 3378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:34.600334 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:34.600338 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:34.600393 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:34.603982 (+ 3589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.604844 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.604849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.604851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.610829 (+ 5978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.611329 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.611333 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.611335 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.611698 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:34.611852 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:34.612427 (+ 575us) server_negotiation.cc:299] Negotiation successful
0808 13:09:34.612697 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 13:09:34.613152 4068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.592053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:33689 (local address 127.0.0.1:40292)
0808 13:09:34.592236 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.592267 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:34.592375 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:34.592739 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:34.592745 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:34.593228 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:34.593375 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:34.593383 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.594728 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.594735 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:34.596012 (+ 1277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:34.596028 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.596959 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.596975 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:34.597193 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:34.597775 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:34.597802 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:34.600143 (+ 2341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:34.604139 (+ 3996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:34.604144 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:34.604147 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:34.604692 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:34.610951 (+ 6259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:34.610955 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:34.610957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:34.611122 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:34.611830 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:34.611835 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:34.612116 (+ 281us) client_negotiation.cc:769] Sending connection context
0808 13:09:34.612276 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 13:09:34.612713 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
Aug 08 13:09:34 dist-test-slave-t3q3 krb5kdc[3945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194572, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.189@KRBTEST.COM
I20260808 13:09:34.619376 4064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad: Bootstrap starting.
Aug 08 13:09:34 dist-test-slave-t3q3 krb5kdc[3945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194574, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.188@KRBTEST.COM for kudu/127.1.124.188@KRBTEST.COM
I20260808 13:09:34.623471 4064 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:34.624924 4064 log.cc:824] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad: Log is configured to *not* fsync() on all Append() calls
I20260808 13:09:34.626829 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.606889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45020 (local address 127.1.124.189:43019)
0808 13:09:34.607114 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:34.607122 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:34.607145 (+ 23us) server_negotiation.cc:407] Connection header received
0808 13:09:34.607207 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:34.607214 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:34.607302 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:34.607424 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:34.608666 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.610195 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.612527 (+ 2332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.613234 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.613373 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:34.619336 (+ 5963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:34.619342 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:34.619348 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:34.619387 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:34.622870 (+ 3483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.623841 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.623850 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.623855 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.624039 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.624580 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.624585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.624590 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.624974 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:34.625265 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:34.626245 (+ 980us) server_negotiation.cc:299] Negotiation successful
0808 13:09:34.626481 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 13:09:34.628392 4295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.604775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:33689 (local address 127.0.0.1:40296)
0808 13:09:34.605108 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.605125 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:34.605228 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:34.612906 (+ 7678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:34.612911 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:34.613769 (+ 858us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:34.614044 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:34.614066 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.615301 (+ 1235us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.615309 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:34.616664 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:34.616682 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.617726 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.617733 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:34.617985 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:34.618687 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:34.618709 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:34.621391 (+ 2682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:34.625195 (+ 3804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:34.625200 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:34.625204 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:34.625713 (+ 509us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:34.626146 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:34.626152 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:34.626159 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:34.626327 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:34.626840 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:34.626844 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:34.626998 (+ 154us) client_negotiation.cc:769] Sending connection context
0808 13:09:34.627152 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 13:09:34.627478 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225}
I20260808 13:09:34.628458 4064 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad: No bootstrap required, opened a new log
I20260808 13:09:34.628428 4297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.605068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40296 (local address 127.1.124.188:33689)
0808 13:09:34.612386 (+ 7318us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:34.612393 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:34.612414 (+ 21us) server_negotiation.cc:407] Connection header received
0808 13:09:34.612458 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:34.612462 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:34.612532 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:34.612633 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:34.614259 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.615165 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.616844 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.617559 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.617696 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:34.621652 (+ 3956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:34.621660 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:34.621664 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:34.621700 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:34.625068 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.625853 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.625860 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.625864 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.626014 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.626429 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.626436 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.626439 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.626717 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:34.626889 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:34.627262 (+ 373us) server_negotiation.cc:299] Negotiation successful
0808 13:09:34.627601 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":155,"server-negotiator.queue_time_us":7192,"thread_start_us":6985,"threads_started":1}
I20260808 13:09:34.632437 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:09:34.633683 4287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68b47c1b5d324c4bb30ad0c0ec5b78b5: Bootstrap starting.
I20260808 13:09:34.636137 4022 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "85505a49c3e64a1a8aed972e438643ad"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49552
I20260808 13:09:34.636687 4287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68b47c1b5d324c4bb30ad0c0ec5b78b5: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:34.637626 4287 log.cc:824] T 00000000000000000000000000000000 P 68b47c1b5d324c4bb30ad0c0ec5b78b5: Log is configured to *not* fsync() on all Append() calls
I20260808 13:09:34.639650 4064 raft_consensus.cc:348] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85505a49c3e64a1a8aed972e438643ad" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44807 } } peers { permanent_uuid: "e72328649fed4f7da247d3b895cb5ccc" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } } peers { permanent_uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } }
I20260808 13:09:34.640026 4064 raft_consensus.cc:729] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85505a49c3e64a1a8aed972e438643ad, State: Initialized, Role: FOLLOWER
I20260808 13:09:34.640234 4287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68b47c1b5d324c4bb30ad0c0ec5b78b5: No bootstrap required, opened a new log
I20260808 13:09:34.640591 4064 consensus_queue.cc:260] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85505a49c3e64a1a8aed972e438643ad" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44807 } } peers { permanent_uuid: "e72328649fed4f7da247d3b895cb5ccc" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } } peers { permanent_uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } }
I20260808 13:09:34.641572 4300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85505a49c3e64a1a8aed972e438643ad" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44807 } } peers { permanent_uuid: "e72328649fed4f7da247d3b895cb5ccc" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } } peers { permanent_uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } } }
I20260808 13:09:34.641996 4300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:09:34.642760 4064 sys_catalog.cc:564] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:09:34.642913 4022 raft_consensus.cc:482] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 13:09:34.643088 4022 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:34.647236 4022 raft_consensus.cc:504] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85505a49c3e64a1a8aed972e438643ad" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44807 } } peers { permanent_uuid: "e72328649fed4f7da247d3b895cb5ccc" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } } peers { permanent_uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } }
I20260808 13:09:34.649796 4287 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68b47c1b5d324c4bb30ad0c0ec5b78b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85505a49c3e64a1a8aed972e438643ad" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44807 } } peers { permanent_uuid: "e72328649fed4f7da247d3b895cb5ccc" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } } peers { permanent_uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } }
I20260808 13:09:34.650228 4287 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68b47c1b5d324c4bb30ad0c0ec5b78b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68b47c1b5d324c4bb30ad0c0ec5b78b5, State: Initialized, Role: FOLLOWER
I20260808 13:09:34.650770 4287 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68b47c1b5d324c4bb30ad0c0ec5b78b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85505a49c3e64a1a8aed972e438643ad" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44807 } } peers { permanent_uuid: "e72328649fed4f7da247d3b895cb5ccc" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } } peers { permanent_uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } }
I20260808 13:09:34.651046 4022 leader_election.cc:288] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad [CANDIDATE]: Term 1 election: Requested vote from peers e72328649fed4f7da247d3b895cb5ccc (127.1.124.189:43019), 68b47c1b5d324c4bb30ad0c0ec5b78b5 (127.1.124.188:33689)
I20260808 13:09:34.653131 3965 master_runner.cc:186] Error getting master registration for 127.1.124.189:43019: OK, instance_id { permanent_uuid: "e72328649fed4f7da247d3b895cb5ccc" instance_seqno: 1786194573990215 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 13:09:34.666190 3965 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 13:09:34.669898 4316 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:09:34.670194 4316 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:09:34.672470 4321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68b47c1b5d324c4bb30ad0c0ec5b78b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85505a49c3e64a1a8aed972e438643ad" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44807 } } peers { permanent_uuid: "e72328649fed4f7da247d3b895cb5ccc" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } } peers { permanent_uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } } }
I20260808 13:09:34.673436 4321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68b47c1b5d324c4bb30ad0c0ec5b78b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:09:34.673807 4287 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68b47c1b5d324c4bb30ad0c0ec5b78b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:09:34.674142 4253 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "85505a49c3e64a1a8aed972e438643ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5"
I20260808 13:09:34.674677 4253 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68b47c1b5d324c4bb30ad0c0ec5b78b5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:34.679255 4253 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 68b47c1b5d324c4bb30ad0c0ec5b78b5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85505a49c3e64a1a8aed972e438643ad in term 1.
I20260808 13:09:34.680099 3988 leader_election.cc:302] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68b47c1b5d324c4bb30ad0c0ec5b78b5, 85505a49c3e64a1a8aed972e438643ad; no voters:
I20260808 13:09:34.680676 4300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad [term 1 FOLLOWER]: Leader election won for term 1
W20260808 13:09:34.685113 3993 tablet.cc:2369] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:09:34.710943 4300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad [term 1 LEADER]: Becoming Leader. State: Replica: 85505a49c3e64a1a8aed972e438643ad, State: Running, Role: LEADER
I20260808 13:09:34.712275 4300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85505a49c3e64a1a8aed972e438643ad" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44807 } } peers { permanent_uuid: "e72328649fed4f7da247d3b895cb5ccc" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } } peers { permanent_uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } }
I20260808 13:09:34.724246 4181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.681387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45036 (local address 127.1.124.189:43019)
0808 13:09:34.681595 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:34.681603 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:34.691934 (+ 10331us) server_negotiation.cc:407] Connection header received
0808 13:09:34.692171 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:34.692180 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:34.692290 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:34.692421 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:34.697899 (+ 5478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.699362 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.703449 (+ 4087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.704215 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.704365 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:34.714401 (+ 10036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:34.714408 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:34.714413 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:34.714456 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:34.719726 (+ 5270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.721039 (+ 1313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.721049 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.721055 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.721303 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.721915 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.721923 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.721929 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.722508 (+ 579us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:34.722831 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:34.723478 (+ 647us) server_negotiation.cc:299] Negotiation successful
0808 13:09:34.723761 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260808 13:09:34.724308 4334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85505a49c3e64a1a8aed972e438643ad. Latest consensus state: current_term: 1 leader_uuid: "85505a49c3e64a1a8aed972e438643ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85505a49c3e64a1a8aed972e438643ad" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44807 } } peers { permanent_uuid: "e72328649fed4f7da247d3b895cb5ccc" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } } peers { permanent_uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } } }
I20260808 13:09:34.724968 4170 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } attempt: 5
I20260808 13:09:34.724869 4334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:34.736785 4345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:09:34.739223 4345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 13:09:34 dist-test-slave-t3q3 krb5kdc[3945](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194573, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.189@KRBTEST.COM for kudu/127.1.124.188@KRBTEST.COM
I20260808 13:09:34.750272 4297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.683140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40302 (local address 127.1.124.188:33689)
0808 13:09:34.683344 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:34.683353 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:34.702168 (+ 18815us) server_negotiation.cc:407] Connection header received
0808 13:09:34.702538 (+ 370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:34.702547 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:34.702648 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:34.702751 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:34.705418 (+ 2667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.706807 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.713392 (+ 6585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.714001 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.714142 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:34.737372 (+ 23230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:34.737380 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:34.737386 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:34.737429 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:34.743664 (+ 6235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.746480 (+ 2816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.746487 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.746490 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.746648 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.748177 (+ 1529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.748187 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.748193 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.748978 (+ 785us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:34.749274 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:34.749622 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 13:09:34.749857 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 13:09:34.753722 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.725498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:33689 (local address 127.0.0.1:40316)
0808 13:09:34.725935 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:34.725960 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:34.726084 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:34.727158 (+ 1074us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:34.727166 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:34.727841 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:34.728153 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:34.728171 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.730136 (+ 1965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.730146 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:34.732176 (+ 2030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:34.732200 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.734073 (+ 1873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.734082 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:34.734450 (+ 368us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:34.735601 (+ 1151us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:34.735632 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:34.740944 (+ 5312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:34.750714 (+ 9770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:34.750722 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:34.750728 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:34.751297 (+ 569us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:34.751758 (+ 461us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:34.751762 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:34.751766 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:34.751940 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:34.752655 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:34.752660 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:34.752833 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 13:09:34.752997 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 13:09:34.753335 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278}
I20260808 13:09:34.754809 4293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.726356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40316 (local address 127.1.124.188:33689)
0808 13:09:34.726595 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:34.726604 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:34.726623 (+ 19us) server_negotiation.cc:407] Connection header received
0808 13:09:34.726696 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:34.726704 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:34.726800 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:34.726918 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:34.728389 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.729891 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.732416 (+ 2525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.733481 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.733666 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:34.745666 (+ 12000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:34.745678 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:34.745685 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:34.745748 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:34.750568 (+ 4820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.751459 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.751465 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.751468 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.751607 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.752111 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.752115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.752118 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.752485 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:34.754037 (+ 1552us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:34.754202 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 13:09:34.754463 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 13:09:34.756258 4253 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 68b47c1b5d324c4bb30ad0c0ec5b78b5 [term 1 FOLLOWER]: Refusing update from remote peer 85505a49c3e64a1a8aed972e438643ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:09:34.764257 4334 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:09:34.770018 4170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e72328649fed4f7da247d3b895cb5ccc: Bootstrap starting.
I20260808 13:09:34.774188 4170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e72328649fed4f7da247d3b895cb5ccc: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:34.775535 4170 log.cc:824] T 00000000000000000000000000000000 P e72328649fed4f7da247d3b895cb5ccc: Log is configured to *not* fsync() on all Append() calls
W20260808 13:09:34.777091 4357 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 68b47c1b5d324c4bb30ad0c0ec5b78b5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:09:34.777243 4357 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:09:34.781432 4294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:34.673975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49556 (local address 127.1.124.190:44807)
0808 13:09:34.674227 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:34.674237 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:34.690813 (+ 16576us) server_negotiation.cc:407] Connection header received
0808 13:09:34.693039 (+ 2226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:34.693045 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:34.693126 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:34.693202 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:34.700166 (+ 6964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.701403 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.714023 (+ 12620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:34.714749 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:34.714856 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:34.737041 (+ 22185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:34.737050 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:34.737057 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:34.737110 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:34.755899 (+ 18789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.756945 (+ 1046us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.756950 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.756954 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.757131 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:34.757618 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:34.757623 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:34.757627 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:34.758011 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:34.780463 (+ 22452us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:34.780693 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 13:09:34.781008 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
W20260808 13:09:34.783118 4223 tablet.cc:2369] T 00000000000000000000000000000000 P 68b47c1b5d324c4bb30ad0c0ec5b78b5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:09:34.795269 4170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e72328649fed4f7da247d3b895cb5ccc: No bootstrap required, opened a new log
I20260808 13:09:34.764520 4186 master_runner.cc:186] Error getting master registration for 127.1.124.189:43019: OK, instance_id { permanent_uuid: "e72328649fed4f7da247d3b895cb5ccc" instance_seqno: 1786194573990215 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 13:09:34.798961 4186 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:09:34.812211 4321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68b47c1b5d324c4bb30ad0c0ec5b78b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85505a49c3e64a1a8aed972e438643ad. Latest consensus state: current_term: 1 leader_uuid: "85505a49c3e64a1a8aed972e438643ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85505a49c3e64a1a8aed972e438643ad" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44807 } } peers { permanent_uuid: "e72328649fed4f7da247d3b895cb5ccc" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } } peers { permanent_uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } } }
I20260808 13:09:34.812551 4321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68b47c1b5d324c4bb30ad0c0ec5b78b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:09:34.818507 4347 mvcc.cc:205] Tried to move back new op lower bound from 7316252978189516800 to 7316252978028257280. Current Snapshot: MvccSnapshot[applied={T|T < 7316252978189516800}]
I20260808 13:09:34.821879 4334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "85505a49c3e64a1a8aed972e438643ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85505a49c3e64a1a8aed972e438643ad" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44807 } } peers { permanent_uuid: "e72328649fed4f7da247d3b895cb5ccc" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } } peers { permanent_uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } } }
I20260808 13:09:34.822259 4334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:34.824147 4345 catalog_manager.cc:1347] Generated new cluster ID: f86be07ca356463bb4f26e23c89a1808
I20260808 13:09:34.824256 4345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:09:34.824983 4321 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68b47c1b5d324c4bb30ad0c0ec5b78b5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "85505a49c3e64a1a8aed972e438643ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85505a49c3e64a1a8aed972e438643ad" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44807 } } peers { permanent_uuid: "e72328649fed4f7da247d3b895cb5ccc" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } } peers { permanent_uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } } }
I20260808 13:09:34.825278 4321 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68b47c1b5d324c4bb30ad0c0ec5b78b5 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:09:34.820173 4170 raft_consensus.cc:348] T 00000000000000000000000000000000 P e72328649fed4f7da247d3b895cb5ccc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85505a49c3e64a1a8aed972e438643ad" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44807 } } peers { permanent_uuid: "e72328649fed4f7da247d3b895cb5ccc" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } } peers { permanent_uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } }
I20260808 13:09:34.826293 4170 raft_consensus.cc:729] T 00000000000000000000000000000000 P e72328649fed4f7da247d3b895cb5ccc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e72328649fed4f7da247d3b895cb5ccc, State: Initialized, Role: FOLLOWER
I20260808 13:09:34.826855 4170 consensus_queue.cc:260] T 00000000000000000000000000000000 P e72328649fed4f7da247d3b895cb5ccc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85505a49c3e64a1a8aed972e438643ad" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44807 } } peers { permanent_uuid: "e72328649fed4f7da247d3b895cb5ccc" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } } peers { permanent_uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } }
I20260808 13:09:34.838093 4365 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72328649fed4f7da247d3b895cb5ccc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85505a49c3e64a1a8aed972e438643ad" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44807 } } peers { permanent_uuid: "e72328649fed4f7da247d3b895cb5ccc" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } } peers { permanent_uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } } }
I20260808 13:09:34.840466 4170 sys_catalog.cc:564] T 00000000000000000000000000000000 P e72328649fed4f7da247d3b895cb5ccc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:09:34.840529 4134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "85505a49c3e64a1a8aed972e438643ad" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "e72328649fed4f7da247d3b895cb5ccc"
I20260808 13:09:34.841054 4134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e72328649fed4f7da247d3b895cb5ccc [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:34.841812 4365 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72328649fed4f7da247d3b895cb5ccc [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:09:34.844921 4134 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e72328649fed4f7da247d3b895cb5ccc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 85505a49c3e64a1a8aed972e438643ad in term 1.
I20260808 13:09:34.846573 4132 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e72328649fed4f7da247d3b895cb5ccc [term 1 FOLLOWER]: Refusing update from remote peer 85505a49c3e64a1a8aed972e438643ad: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:09:34.847637 4300 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad [LEADER]: Connected to new peer: Peer: permanent_uuid: "e72328649fed4f7da247d3b895cb5ccc" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:09:34.869020 4345 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:09:34.872308 4345 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:09:34.883379 4365 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72328649fed4f7da247d3b895cb5ccc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 85505a49c3e64a1a8aed972e438643ad. Latest consensus state: current_term: 1 leader_uuid: "85505a49c3e64a1a8aed972e438643ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85505a49c3e64a1a8aed972e438643ad" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44807 } } peers { permanent_uuid: "e72328649fed4f7da247d3b895cb5ccc" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } } peers { permanent_uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } } }
I20260808 13:09:34.883719 4365 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72328649fed4f7da247d3b895cb5ccc [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:09:34.884819 4334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "85505a49c3e64a1a8aed972e438643ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85505a49c3e64a1a8aed972e438643ad" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44807 } } peers { permanent_uuid: "e72328649fed4f7da247d3b895cb5ccc" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } } peers { permanent_uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } } }
I20260808 13:09:34.885104 4334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:34.885571 4373 sys_catalog.cc:455] T 00000000000000000000000000000000 P e72328649fed4f7da247d3b895cb5ccc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "85505a49c3e64a1a8aed972e438643ad" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "85505a49c3e64a1a8aed972e438643ad" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 44807 } } peers { permanent_uuid: "e72328649fed4f7da247d3b895cb5ccc" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 43019 } } peers { permanent_uuid: "68b47c1b5d324c4bb30ad0c0ec5b78b5" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 33689 } } }
I20260808 13:09:34.886569 4373 sys_catalog.cc:458] T 00000000000000000000000000000000 P e72328649fed4f7da247d3b895cb5ccc [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:09:34.891004 4345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 85505a49c3e64a1a8aed972e438643ad: Generated new TSK 0
I20260808 13:09:34.892208 4345 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:09:34.914407 4390 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260808 13:09:35.003531 4294 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:34.988588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49562 (local address 127.1.124.190:44807)
0808 13:09:34.988853 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:34.988862 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:35.001099 (+ 12237us) server_negotiation.cc:407] Connection header received
0808 13:09:35.001352 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:35.001361 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:35.001468 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:35.001584 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:35.003275 (+ 1691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49562: BlockingRecv error: recv got EOF from 127.0.0.1:49562 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
W20260808 13:09:35.005168 4297 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:34.988260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40328 (local address 127.1.124.188:33689)
0808 13:09:34.988489 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:34.988496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:35.001904 (+ 13408us) server_negotiation.cc:407] Connection header received
0808 13:09:35.002090 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:35.002099 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:35.002213 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:35.002341 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:35.004839 (+ 2498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40328: BlockingRecv error: recv got EOF from 127.0.0.1:40328 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
W20260808 13:09:35.006376 4181 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:34.989039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45048 (local address 127.1.124.189:43019)
0808 13:09:34.989284 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:34.989292 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:35.004828 (+ 15536us) server_negotiation.cc:407] Connection header received
0808 13:09:35.004975 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:35.004980 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:35.005064 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:35.005189 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:35.006171 (+ 982us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45048: BlockingRecv error: recv got EOF from 127.0.0.1:45048 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
W20260808 13:09:35.043831 4181 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:35.040380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45052 (local address 127.1.124.189:43019)
0808 13:09:35.040571 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:35.040577 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:35.042120 (+ 1543us) server_negotiation.cc:407] Connection header received
0808 13:09:35.042386 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:35.042390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:35.042484 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:35.042619 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:35.043527 (+ 908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45052: BlockingRecv error: recv got EOF from 127.0.0.1:45052 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260808 13:09:35.056265 4181 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:35.054065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45060 (local address 127.1.124.189:43019)
0808 13:09:35.054287 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:35.054297 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:35.054821 (+ 524us) server_negotiation.cc:407] Connection header received
0808 13:09:35.055054 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:35.055060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:35.055141 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:35.055252 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:35.056024 (+ 772us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45060: BlockingRecv error: recv got EOF from 127.0.0.1:45060 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260808 13:09:35.063445 4181 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:35.061387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45070 (local address 127.1.124.189:43019)
0808 13:09:35.061609 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:35.061618 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:35.062099 (+ 481us) server_negotiation.cc:407] Connection header received
0808 13:09:35.062321 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:35.062326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:35.062404 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:35.062512 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:35.063216 (+ 704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45070: BlockingRecv error: recv got EOF from 127.0.0.1:45070 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 13:09:35.072918 4181 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:35.070625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45074 (local address 127.1.124.189:43019)
0808 13:09:35.070853 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:35.070862 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:35.071367 (+ 505us) server_negotiation.cc:407] Connection header received
0808 13:09:35.071580 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:35.071585 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:35.071667 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:35.071783 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:35.072686 (+ 903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45074: BlockingRecv error: recv got EOF from 127.0.0.1:45074 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260808 13:09:35.088300 4181 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:35.085987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45076 (local address 127.1.124.189:43019)
0808 13:09:35.086225 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:35.086235 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:35.086814 (+ 579us) server_negotiation.cc:407] Connection header received
0808 13:09:35.087047 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:35.087051 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:35.087122 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:35.087238 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:35.088003 (+ 765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45076: BlockingRecv error: recv got EOF from 127.0.0.1:45076 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 13:09:35.110559 4181 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:35.107929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45082 (local address 127.1.124.189:43019)
0808 13:09:35.108196 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:35.108206 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:35.108783 (+ 577us) server_negotiation.cc:407] Connection header received
0808 13:09:35.109160 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:35.109165 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:35.109260 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:35.109422 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:35.110271 (+ 849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45082: BlockingRecv error: recv got EOF from 127.0.0.1:45082 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260808 13:09:35.154279 4181 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:35.146812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45098 (local address 127.1.124.189:43019)
0808 13:09:35.147057 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:35.147067 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:35.152571 (+ 5504us) server_negotiation.cc:407] Connection header received
0808 13:09:35.152865 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:35.152870 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:35.153010 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:35.153118 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:35.153980 (+ 862us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45098: BlockingRecv error: recv got EOF from 127.0.0.1:45098 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260808 13:09:35.225333 4181 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:35.222904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45104 (local address 127.1.124.189:43019)
0808 13:09:35.223227 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:35.223238 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:35.223831 (+ 593us) server_negotiation.cc:407] Connection header received
0808 13:09:35.224111 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:35.224121 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:35.224197 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:35.224316 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:35.225116 (+ 800us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45104: BlockingRecv error: recv got EOF from 127.0.0.1:45104 (error 108)
Metrics: {"server-negotiator.queue_time_us":128}
W20260808 13:09:35.364895 4181 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:35.362873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45114 (local address 127.1.124.189:43019)
0808 13:09:35.363144 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:35.363151 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:35.363431 (+ 280us) server_negotiation.cc:407] Connection header received
0808 13:09:35.363588 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:35.363592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:35.363680 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:35.363774 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:35.364640 (+ 866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45114: BlockingRecv error: recv got EOF from 127.0.0.1:45114 (error 108)
Metrics: {"server-negotiator.queue_time_us":117}
W20260808 13:09:35.626919 4181 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:35.624968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45126 (local address 127.1.124.189:43019)
0808 13:09:35.625237 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:35.625244 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:35.625413 (+ 169us) server_negotiation.cc:407] Connection header received
0808 13:09:35.625697 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:35.625702 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:35.625775 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:35.625894 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:35.626696 (+ 802us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45126: BlockingRecv error: recv got EOF from 127.0.0.1:45126 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
I20260808 13:09:35.779192 4357 catalog_manager.cc:1259] Loaded cluster ID: f86be07ca356463bb4f26e23c89a1808
I20260808 13:09:35.779340 4357 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 68b47c1b5d324c4bb30ad0c0ec5b78b5: loading cluster ID for follower catalog manager: success
I20260808 13:09:35.781890 4357 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 68b47c1b5d324c4bb30ad0c0ec5b78b5: acquiring CA information for follower catalog manager: success
I20260808 13:09:35.783643 4357 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 68b47c1b5d324c4bb30ad0c0ec5b78b5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 13:09:35.919236 4390 catalog_manager.cc:1259] Loaded cluster ID: f86be07ca356463bb4f26e23c89a1808
I20260808 13:09:35.919391 4390 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e72328649fed4f7da247d3b895cb5ccc: loading cluster ID for follower catalog manager: success
I20260808 13:09:35.922652 4390 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e72328649fed4f7da247d3b895cb5ccc: acquiring CA information for follower catalog manager: success
I20260808 13:09:35.924755 4390 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e72328649fed4f7da247d3b895cb5ccc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 13:09:36.150442 4491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:36.143717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45138 (local address 127.1.124.189:43019)
0808 13:09:36.144584 (+ 867us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:36.144589 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:36.144609 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:09:36.144677 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:36.144681 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:36.144827 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:36.144977 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:09:36.145508 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:36.146543 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:36.147923 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:36.148466 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:36.148624 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:36.149154 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:09:36.149780 (+ 626us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:09:36.149960 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:36.150014 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 13:09:36.150095 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":546,"threads_started":1}
I20260808 13:09:36.179080 4500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:36.157308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40338 (local address 127.1.124.188:33689)
0808 13:09:36.172440 (+ 15132us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:36.172445 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:36.172470 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:09:36.172579 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:36.172584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:36.172753 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:36.172934 (+ 181us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:09:36.173649 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:36.174443 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:36.176556 (+ 2113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:36.176954 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:36.177051 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:36.177693 (+ 642us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:09:36.178413 (+ 720us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:09:36.178537 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:36.178704 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 13:09:36.178772 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14998,"thread_start_us":15727,"threads_started":1}
I20260808 13:09:36.198010 4491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:36.186431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45150 (local address 127.1.124.189:43019)
0808 13:09:36.186595 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:36.186603 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:36.189493 (+ 2890us) server_negotiation.cc:407] Connection header received
0808 13:09:36.189737 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:36.189750 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:36.189865 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:36.189963 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:09:36.191122 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:36.192396 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:36.195447 (+ 3051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:36.196198 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:36.196346 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:36.197053 (+ 707us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:09:36.197216 (+ 163us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:09:36.197301 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:36.197572 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 13:09:36.197666 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 13:09:36.200331 4500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:36.187408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40342 (local address 127.1.124.188:33689)
0808 13:09:36.192710 (+ 5302us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:36.192717 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:36.193292 (+ 575us) server_negotiation.cc:407] Connection header received
0808 13:09:36.193580 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:36.193586 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:36.193691 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:36.193758 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:09:36.194599 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:36.195662 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:36.197762 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:36.198382 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:36.198482 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:36.199454 (+ 972us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:09:36.199580 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:09:36.199660 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:36.199888 (+ 228us) server_negotiation.cc:299] Negotiation successful
0808 13:09:36.199974 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5176}
I20260808 13:09:36.202289 4505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:36.186301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49572 (local address 127.1.124.190:44807)
0808 13:09:36.192548 (+ 6247us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:36.192555 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:36.192574 (+ 19us) server_negotiation.cc:407] Connection header received
0808 13:09:36.192689 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:36.192694 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:36.192951 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:36.193179 (+ 228us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:09:36.195482 (+ 2303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:36.196601 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:36.199431 (+ 2830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:36.200268 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:36.200411 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:36.200958 (+ 547us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:09:36.201469 (+ 511us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:09:36.201586 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:36.201750 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 13:09:36.201835 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6117,"thread_start_us":6185,"threads_started":1}
I20260808 13:09:36.207134 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 3965
I20260808 13:09:36.209061 4054 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:09:36.321269 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 3965
I20260808 13:09:36.352416 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 4076
I20260808 13:09:36.353683 4162 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:09:36.461162 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 4076
I20260808 13:09:36.489812 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 4186
I20260808 13:09:36.491065 4281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:09:36.600595 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 4186
2026-08-08T13:09:36Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5325 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 13:09:36 dist-test-slave-t3q3 krb5kdc[4518](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 13:09:36 dist-test-slave-t3q3 krb5kdc[4518](info): set up 2 sockets
Aug 08 13:09:36 dist-test-slave-t3q3 krb5kdc[4518](info): commencing operation
krb5kdc: starting...
W20260808 13:09:37.971426 1522 mini_kdc.cc:121] Time spent starting KDC: real 1.322s user 0.002s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/test-user.keytab.
Aug 08 13:09:38 dist-test-slave-t3q3 krb5kdc[4518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T13:09:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T13:09:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.1.124.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
Entry for principal kudu/127.1.124.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab.
I20260808 13:09:38.339831 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:39499
--webserver_interface=127.1.124.190
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:45295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:39499,127.1.124.189:45787,127.1.124.188:39145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 13:09:38.697690 4534 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 13:09:38.698033 4534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:38.698112 4534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:38.698199 4534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:38.705711 4534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:09:38.705825 4534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:38.705888 4534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:38.705940 4534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:09:38.706014 4534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:09:38.716473 4534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:45295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:39499,127.1.124.189:45787,127.1.124.188:39145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.190.keytab
--principal=kudu/127.1.124.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.190:39499
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.1.124.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:38.717814 4534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:38.719244 4534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:38.732378 4543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:38.732694 4545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:38.733399 4541 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:38.734107 4534 server_base.cc:1034] running on GCE node
I20260808 13:09:38.735286 4534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:38.737232 4534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:38.738516 4534 hybrid_clock.cc:648] HybridClock initialized: now 1786194578738399 us; error 60 us; skew 500 ppm
I20260808 13:09:38.738976 4534 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:38 dist-test-slave-t3q3 krb5kdc[4518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194578, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:38.746083 4534 init.cc:377] Logged in from keytab as kudu/127.1.124.190@KRBTEST.COM (short username kudu)
I20260808 13:09:38.747606 4534 webserver.cc:460] Webserver started at http://127.1.124.190:44759/ using document root <none> and password file <none>
I20260808 13:09:38.748448 4534 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:38.748545 4534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:38.748946 4534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:38.752126 4534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-0/data/instance:
uuid: "11b6f813a9d84142b8ac90ec86bdff99"
format_stamp: "Formatted at 2026-08-08 13:09:38 on dist-test-slave-t3q3"
server_key: "e6130ec19ed76ab86d9abc4b28b94ca4"
server_key_iv: "28a3d30ea929ff6308e228be28fe2285"
server_key_version: "encryptionkey@0"
I20260808 13:09:38.753031 4534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-0/wal/instance:
uuid: "11b6f813a9d84142b8ac90ec86bdff99"
format_stamp: "Formatted at 2026-08-08 13:09:38 on dist-test-slave-t3q3"
server_key: "e6130ec19ed76ab86d9abc4b28b94ca4"
server_key_iv: "28a3d30ea929ff6308e228be28fe2285"
server_key_version: "encryptionkey@0"
I20260808 13:09:38.758414 4534 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:09:38.762115 4558 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:38.763455 4534 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 13:09:38.763659 4534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-0/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-0/wal
uuid: "11b6f813a9d84142b8ac90ec86bdff99"
format_stamp: "Formatted at 2026-08-08 13:09:38 on dist-test-slave-t3q3"
server_key: "e6130ec19ed76ab86d9abc4b28b94ca4"
server_key_iv: "28a3d30ea929ff6308e228be28fe2285"
server_key_version: "encryptionkey@0"
I20260808 13:09:38.763860 4534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:38.783262 4534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:38.787911 4534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:38.788251 4534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:38.808207 4534 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.190:39499
I20260808 13:09:38.808244 4640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.190:39499 every 8 connection(s)
I20260808 13:09:38.809562 4534 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-0/data/info.pb
I20260808 13:09:38.813288 4641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:38.815609 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 4534
I20260808 13:09:38.815804 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-0/wal/instance
I20260808 13:09:38.816373 1522 external_mini_cluster.cc:1428] Setting key cc3924ebb4fd409247b096610293668e
I20260808 13:09:38.822551 4641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39499 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:09:38 dist-test-slave-t3q3 krb5kdc[4518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194578, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:09:38.848260 4643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:38.826603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47618 (local address 127.1.124.190:39499)
0808 13:09:38.827526 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:38.827542 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:38.828119 (+ 577us) server_negotiation.cc:407] Connection header received
0808 13:09:38.830301 (+ 2182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:38.830311 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:38.830626 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:38.830864 (+ 238us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:38.832097 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:38.834214 (+ 2117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:38.835619 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:38.836604 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:38.836795 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:38.840448 (+ 3653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:38.840461 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:38.840481 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:38.840522 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:38.843761 (+ 3239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:38.844553 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:38.844570 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:38.844572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:38.844720 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:38.845124 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:38.845136 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:38.845138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:38.845618 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:38.845845 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:38.846412 (+ 567us) server_negotiation.cc:299] Negotiation successful
0808 13:09:38.846749 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":665,"thread_start_us":406,"threads_started":1}
I20260808 13:09:38.848295 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:38.826603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39499 (local address 127.0.0.1:47618)
0808 13:09:38.827910 (+ 1307us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:38.827953 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:38.829913 (+ 1960us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:38.830894 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:38.830899 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:38.831481 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:38.831825 (+ 344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:38.831843 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:38.834377 (+ 2534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:38.834381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:38.835457 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:38.835469 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:38.836839 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:38.836844 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:38.836967 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:38.837677 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:38.837698 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:38.840206 (+ 2508us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:38.843910 (+ 3704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:38.843921 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:38.843933 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:38.844424 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:38.844833 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:38.844840 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:38.844845 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:38.845008 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:38.845763 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:38.845776 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:38.846007 (+ 231us) client_negotiation.cc:769] Sending connection context
0808 13:09:38.846254 (+ 247us) client_negotiation.cc:240] Negotiation successful
0808 13:09:38.846880 (+ 626us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1052,"mutex_wait_us":169,"spinlock_wait_cycles":63744,"thread_start_us":809,"threads_started":1}
WARNING: no policy specified for kudu/127.1.124.189@KRBTEST.COM; defaulting to no policy
I20260808 13:09:38.851255 4641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:09:38.852391 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:38.851948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:45787 (local address 127.0.0.1:49216)
0808 13:09:38.852161 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:38.852275 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:45787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260808 13:09:38.852715 4559 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.124.189:45787: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.189@KRBTEST.COM" created.
I20260808 13:09:38.854437 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:38.854071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:45787 (local address 127.0.0.1:49220)
0808 13:09:38.854224 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:38.854301 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:45787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 13:09:38.854696 4641 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.189:45787: Network error: Client connection negotiation failed: client connection to 127.1.124.189:45787: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.189.keytab.
Entry for principal kudu/127.1.124.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.189.keytab.
I20260808 13:09:38.893802 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.189:45787
--webserver_interface=127.1.124.189
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.189.keytab
--principal=kudu/127.1.124.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:45295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:39499,127.1.124.189:45787,127.1.124.188:39145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:09:38.904147 4641 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } attempt: 1
I20260808 13:09:38.905438 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:38.904973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:45787 (local address 127.0.0.1:49236)
0808 13:09:38.905189 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:38.905308 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:45787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260808 13:09:38.906504 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:38.906194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:45787 (local address 127.0.0.1:49248)
0808 13:09:38.906323 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:38.906386 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:45787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 13:09:38.906770 4641 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.189:45787: Network error: Client connection negotiation failed: client connection to 127.1.124.189:45787: connect: Connection refused (error 111)
I20260808 13:09:38.975063 4641 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } attempt: 2
I20260808 13:09:38.976509 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:38.975871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:45787 (local address 127.0.0.1:49256)
0808 13:09:38.976199 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:38.976330 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:45787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260808 13:09:38.977660 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:38.977257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:45787 (local address 127.0.0.1:49266)
0808 13:09:38.977426 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:38.977508 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:45787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 13:09:38.977885 4641 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.189:45787: Network error: Client connection negotiation failed: client connection to 127.1.124.189:45787: connect: Connection refused (error 111)
I20260808 13:09:39.069155 4641 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } attempt: 3
I20260808 13:09:39.070416 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.069963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:45787 (local address 127.0.0.1:49278)
0808 13:09:39.070179 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.070279 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:45787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260808 13:09:39.071533 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.071140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:45787 (local address 127.0.0.1:49284)
0808 13:09:39.071306 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.071376 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:45787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260808 13:09:39.071727 4641 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.189:45787: Network error: Client connection negotiation failed: client connection to 127.1.124.189:45787: connect: Connection refused (error 111)
I20260808 13:09:39.215039 4641 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } attempt: 4
I20260808 13:09:39.216468 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.215842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:45787 (local address 127.0.0.1:49290)
0808 13:09:39.216230 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.216323 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:45787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223}
I20260808 13:09:39.217747 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.217317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:45787 (local address 127.0.0.1:49306)
0808 13:09:39.217495 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.217603 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.189:45787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260808 13:09:39.217944 4641 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.189:45787: Network error: Client connection negotiation failed: client connection to 127.1.124.189:45787: connect: Connection refused (error 111)
W20260808 13:09:39.246376 4649 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 13:09:39.246783 4649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:39.246915 4649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:39.247064 4649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:39.254590 4649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:09:39.254724 4649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:39.254828 4649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:39.254926 4649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:09:39.255018 4649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:09:39.265027 4649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:45295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:39499,127.1.124.189:45787,127.1.124.188:39145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.189.keytab
--principal=kudu/127.1.124.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.189:45787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.1.124.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:39.266453 4649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:39.267839 4649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:39.280032 4656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:39.280150 4657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:39.280174 4659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:39.281283 4649 server_base.cc:1034] running on GCE node
I20260808 13:09:39.282038 4649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:39.283430 4649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:39.284677 4649 hybrid_clock.cc:648] HybridClock initialized: now 1786194579284607 us; error 42 us; skew 500 ppm
I20260808 13:09:39.285005 4649 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:39 dist-test-slave-t3q3 krb5kdc[4518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194579, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:39.288470 4649 init.cc:377] Logged in from keytab as kudu/127.1.124.189@KRBTEST.COM (short username kudu)
I20260808 13:09:39.290086 4649 webserver.cc:460] Webserver started at http://127.1.124.189:41589/ using document root <none> and password file <none>
I20260808 13:09:39.290818 4649 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:39.290913 4649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:39.291203 4649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:39.294229 4649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-1/data/instance:
uuid: "1f18f53a26a7435fb2b7dffae1cede93"
format_stamp: "Formatted at 2026-08-08 13:09:39 on dist-test-slave-t3q3"
server_key: "de70b2c8c01ca21fd90121604482c36c"
server_key_iv: "d127a88ab9c5b0ea627848e1e86d8774"
server_key_version: "encryptionkey@0"
I20260808 13:09:39.295030 4649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-1/wal/instance:
uuid: "1f18f53a26a7435fb2b7dffae1cede93"
format_stamp: "Formatted at 2026-08-08 13:09:39 on dist-test-slave-t3q3"
server_key: "de70b2c8c01ca21fd90121604482c36c"
server_key_iv: "d127a88ab9c5b0ea627848e1e86d8774"
server_key_version: "encryptionkey@0"
I20260808 13:09:39.300706 4649 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260808 13:09:39.304492 4669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:39.305972 4649 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 13:09:39.306170 4649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-1/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-1/wal
uuid: "1f18f53a26a7435fb2b7dffae1cede93"
format_stamp: "Formatted at 2026-08-08 13:09:39 on dist-test-slave-t3q3"
server_key: "de70b2c8c01ca21fd90121604482c36c"
server_key_iv: "d127a88ab9c5b0ea627848e1e86d8774"
server_key_version: "encryptionkey@0"
I20260808 13:09:39.306412 4649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:39.318805 4649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:39.323184 4649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:39.323484 4649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:39.347744 4649 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.189:45787
I20260808 13:09:39.347772 4742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.189:45787 every 8 connection(s)
I20260808 13:09:39.349291 4649 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-1/data/info.pb
I20260808 13:09:39.353302 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 4649
I20260808 13:09:39.353514 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-1/wal/instance
I20260808 13:09:39.354035 1522 external_mini_cluster.cc:1428] Setting key f45a98e2ea368835f32b0b4a6ea8e946
I20260808 13:09:39.354305 4743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:39.364825 4743 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39499 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:09:39 dist-test-slave-t3q3 krb5kdc[4518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194579, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.189@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.1.124.188@KRBTEST.COM; defaulting to no policy
I20260808 13:09:39.394719 4749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.368669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57874 (local address 127.1.124.190:39499)
0808 13:09:39.369303 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:39.369311 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:39.370507 (+ 1196us) server_negotiation.cc:407] Connection header received
0808 13:09:39.373589 (+ 3082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:39.373597 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:39.373693 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:39.373776 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:39.376400 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.377681 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.380294 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.381120 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.381252 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:39.386458 (+ 5206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:39.386468 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:39.386476 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:39.386530 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:39.390711 (+ 4181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:39.391812 (+ 1101us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:39.391820 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:39.391826 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:39.392004 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:39.392574 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:39.392582 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:39.392588 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:39.393100 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:39.393209 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:39.394085 (+ 876us) server_negotiation.cc:299] Negotiation successful
0808 13:09:39.394310 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":301,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.1.124.188@KRBTEST.COM" created.
I20260808 13:09:39.395915 4750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.369409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39499 (local address 127.0.0.1:57874)
0808 13:09:39.370335 (+ 926us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.370385 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:39.373262 (+ 2877us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:39.374382 (+ 1120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:39.374395 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:39.375224 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:39.376146 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:39.376177 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.377933 (+ 1756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.377941 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:39.380127 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:39.380146 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.381619 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.381628 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:39.381959 (+ 331us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:39.382826 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:39.382859 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:39.386188 (+ 3329us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:39.390901 (+ 4713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:39.390911 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:39.390932 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:39.391636 (+ 704us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:39.392203 (+ 567us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:39.392208 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:39.392212 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:39.392431 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:39.393295 (+ 864us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:39.393302 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:39.393647 (+ 345us) client_negotiation.cc:769] Sending connection context
0808 13:09:39.393996 (+ 349us) client_negotiation.cc:240] Negotiation successful
0808 13:09:39.394529 (+ 533us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"thread_start_us":371,"threads_started":1}
I20260808 13:09:39.397724 4743 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:09:39 dist-test-slave-t3q3 krb5kdc[4518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194579, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.189@KRBTEST.COM for kudu/127.1.124.189@KRBTEST.COM
I20260808 13:09:39.415557 4750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.398608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:45787 (local address 127.0.0.1:39348)
0808 13:09:39.398776 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.398799 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:39.398916 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:39.400223 (+ 1307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:39.400227 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:39.400689 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:39.400827 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:39.400835 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.402232 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.402238 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:39.403104 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:39.403116 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.403986 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.403993 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:39.404202 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:39.404849 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:39.404867 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:39.406944 (+ 2077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:39.412718 (+ 5774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:39.412724 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:39.412732 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:39.413150 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:39.413554 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:39.413560 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:39.413563 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:39.413714 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:39.414342 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:39.414346 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:39.414545 (+ 199us) client_negotiation.cc:769] Sending connection context
0808 13:09:39.414710 (+ 165us) client_negotiation.cc:240] Negotiation successful
0808 13:09:39.415046 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260808 13:09:39.415848 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.398924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39348 (local address 127.1.124.189:45787)
0808 13:09:39.399756 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:39.399762 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:39.399790 (+ 28us) server_negotiation.cc:407] Connection header received
0808 13:09:39.399834 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:39.399840 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:39.400028 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:39.400243 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:39.401009 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.402085 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.403250 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.403815 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.403969 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:39.407180 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:39.407192 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:39.407206 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:39.407235 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:39.412580 (+ 5345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:39.413307 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:39.413314 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:39.413316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:39.413432 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:39.413833 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:39.413840 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:39.413842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:39.414210 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:39.414432 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:39.414966 (+ 534us) server_negotiation.cc:299] Negotiation successful
0808 13:09:39.415289 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":675,"thread_start_us":478,"threads_started":1}
I20260808 13:09:39.425876 4743 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:09:39.427258 4750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.426744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:39145 (local address 127.0.0.1:50656)
0808 13:09:39.426959 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.427089 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260808 13:09:39.427645 4677 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
I20260808 13:09:39.429427 4750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.429043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:39145 (local address 127.0.0.1:50662)
0808 13:09:39.429210 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.429299 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260808 13:09:39.429718 4743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:39145: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.1.124.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.188.keytab.
Entry for principal kudu/127.1.124.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.188.keytab.
I20260808 13:09:39.443873 1522 external_mini_cluster.cc:1326] Running /tmp/dist-test-task9J1ueI/build/asan/bin/kudu
/tmp/dist-test-task9J1ueI/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.188:39145
--webserver_interface=127.1.124.188
--webserver_port=0
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.188.keytab
--principal=kudu/127.1.124.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.1.124.148:45295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.1.124.190:39499,127.1.124.189:45787,127.1.124.188:39145
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 13:09:39.479067 4743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } attempt: 1
I20260808 13:09:39.480367 4750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.479854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:39145 (local address 127.0.0.1:50678)
0808 13:09:39.480102 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.480214 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 13:09:39.481384 4750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.481068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:39145 (local address 127.0.0.1:50680)
0808 13:09:39.481212 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.481283 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 13:09:39.481638 4743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:39145: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
I20260808 13:09:39.517195 4641 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } attempt: 5
Aug 08 13:09:39 dist-test-slave-t3q3 krb5kdc[4518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194578, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for kudu/127.1.124.189@KRBTEST.COM
I20260808 13:09:39.531876 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.517948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:45787 (local address 127.0.0.1:39362)
0808 13:09:39.518205 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.518230 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:39.518370 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:39.518763 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:39.518768 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:39.519295 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:39.519545 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:39.519558 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.520700 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.520706 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:39.522092 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:39.522110 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.522875 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.522881 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:39.523111 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:39.523801 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:39.523825 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:39.526261 (+ 2436us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:39.529371 (+ 3110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:39.529377 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:39.529382 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:39.529764 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:39.530162 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:39.530167 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:39.530172 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:39.530306 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:39.530853 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:39.530858 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:39.531017 (+ 159us) client_negotiation.cc:769] Sending connection context
0808 13:09:39.531175 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 13:09:39.531504 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260808 13:09:39.531875 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.518226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39362 (local address 127.1.124.189:45787)
0808 13:09:39.518401 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:39.518406 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:39.518431 (+ 25us) server_negotiation.cc:407] Connection header received
0808 13:09:39.518488 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:39.518492 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:39.518594 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:39.518682 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:39.519734 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.520555 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.522264 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.522714 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.522815 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:39.526466 (+ 3651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:39.526471 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:39.526475 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:39.526508 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:39.529223 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:39.529895 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:39.529908 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:39.529911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:39.530028 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:39.530421 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:39.530426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:39.530430 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:39.530729 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:39.530890 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:39.531288 (+ 398us) server_negotiation.cc:299] Negotiation successful
0808 13:09:39.531506 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 13:09:39.533190 4641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:09:39.534226 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.533828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:39145 (local address 127.0.0.1:50696)
0808 13:09:39.534002 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.534084 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260808 13:09:39.535137 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.534817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:39145 (local address 127.0.0.1:50698)
0808 13:09:39.534978 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.535039 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 13:09:39.535346 4641 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:39145: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
I20260808 13:09:39.549873 4743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } attempt: 2
I20260808 13:09:39.551000 4750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.550560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:39145 (local address 127.0.0.1:50710)
0808 13:09:39.550762 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.550850 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260808 13:09:39.552006 4750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.551718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:39145 (local address 127.0.0.1:50718)
0808 13:09:39.551841 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.551903 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 13:09:39.552278 4743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:39145: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
I20260808 13:09:39.586576 4641 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } attempt: 1
I20260808 13:09:39.587760 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.587287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:39145 (local address 127.0.0.1:50724)
0808 13:09:39.587488 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.587588 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 13:09:39.588768 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.588449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:39145 (local address 127.0.0.1:50728)
0808 13:09:39.588597 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.588671 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 13:09:39.588972 4641 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:39145: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
I20260808 13:09:39.643545 4743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } attempt: 3
I20260808 13:09:39.644948 4750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.644412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:39145 (local address 127.0.0.1:50744)
0808 13:09:39.644678 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.644774 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260808 13:09:39.646066 4750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.645633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:39145 (local address 127.0.0.1:50756)
0808 13:09:39.645872 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.645947 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
W20260808 13:09:39.646284 4743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:39145: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
I20260808 13:09:39.657186 4641 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } attempt: 2
I20260808 13:09:39.658427 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.657878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:39145 (local address 127.0.0.1:50768)
0808 13:09:39.658164 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.658263 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260808 13:09:39.659425 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.659079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:39145 (local address 127.0.0.1:50774)
0808 13:09:39.659228 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.659306 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 13:09:39.659632 4641 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:39145: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
I20260808 13:09:39.765843 4641 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } attempt: 3
I20260808 13:09:39.767223 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.766589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:39145 (local address 127.0.0.1:50788)
0808 13:09:39.766867 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.766988 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260808 13:09:39.768381 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.767968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:39145 (local address 127.0.0.1:50790)
0808 13:09:39.768184 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.768267 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260808 13:09:39.768720 4641 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:39145: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
I20260808 13:09:39.789534 4743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } attempt: 4
I20260808 13:09:39.790762 4750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.790249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:39145 (local address 127.0.0.1:50804)
0808 13:09:39.790513 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.790602 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260808 13:09:39.791975 4750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.791616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:39145 (local address 127.0.0.1:50820)
0808 13:09:39.791785 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.791858 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 13:09:39.792296 4743 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.1.124.188:39145: Network error: Client connection negotiation failed: client connection to 127.1.124.188:39145: connect: Connection refused (error 111)
W20260808 13:09:39.818467 4755 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 13:09:39.818874 4755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 13:09:39.819007 4755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 13:09:39.819175 4755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 13:09:39.827596 4755 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 13:09:39.827699 4755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 13:09:39.827811 4755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 13:09:39.827886 4755 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 13:09:39.827987 4755 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 13:09:39.844753 4755 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.1.124.148:45295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.1.124.190:39499,127.1.124.189:45787,127.1.124.188:39145
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/krb5kdc/kudu_127.1.124.188.keytab
--principal=kudu/127.1.124.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.1.124.188:39145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.1.124.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 08 Aug 2026 12:05:15 UTC on ccycloud.quasar-wzjnwn.root.comops.site
build id 4574795
ASAN enabled
I20260808 13:09:39.846243 4755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 13:09:39.847721 4755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 13:09:39.860208 4765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:39.860298 4764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 13:09:39.860267 4767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 13:09:39.861474 4755 server_base.cc:1034] running on GCE node
I20260808 13:09:39.862336 4755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 13:09:39.863842 4755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 13:09:39.865079 4755 hybrid_clock.cc:648] HybridClock initialized: now 1786194579865009 us; error 35 us; skew 500 ppm
I20260808 13:09:39.865459 4755 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 08 13:09:39 dist-test-slave-t3q3 krb5kdc[4518](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194579, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 13:09:39.869057 4755 init.cc:377] Logged in from keytab as kudu/127.1.124.188@KRBTEST.COM (short username kudu)
I20260808 13:09:39.870662 4755 webserver.cc:460] Webserver started at http://127.1.124.188:34403/ using document root <none> and password file <none>
I20260808 13:09:39.871433 4755 fs_manager.cc:359] Metadata directory not provided
I20260808 13:09:39.871592 4755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 13:09:39.871961 4755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 13:09:39.875396 4755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-2/data/instance:
uuid: "9c749554f46c40898177dcb0dc176ffa"
format_stamp: "Formatted at 2026-08-08 13:09:39 on dist-test-slave-t3q3"
server_key: "9e04d0f7ee6c5ec60ed50df491fd9347"
server_key_iv: "4ee75273fdea93c89cbc4faa251fecab"
server_key_version: "encryptionkey@0"
I20260808 13:09:39.876385 4755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-2/wal/instance:
uuid: "9c749554f46c40898177dcb0dc176ffa"
format_stamp: "Formatted at 2026-08-08 13:09:39 on dist-test-slave-t3q3"
server_key: "9e04d0f7ee6c5ec60ed50df491fd9347"
server_key_iv: "4ee75273fdea93c89cbc4faa251fecab"
server_key_version: "encryptionkey@0"
I20260808 13:09:39.882087 4755 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260808 13:09:39.885872 4778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 13:09:39.887301 4755 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 13:09:39.887511 4755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-2/data,/tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-2/wal
uuid: "9c749554f46c40898177dcb0dc176ffa"
format_stamp: "Formatted at 2026-08-08 13:09:39 on dist-test-slave-t3q3"
server_key: "9e04d0f7ee6c5ec60ed50df491fd9347"
server_key_iv: "4ee75273fdea93c89cbc4faa251fecab"
server_key_version: "encryptionkey@0"
I20260808 13:09:39.887769 4755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 13:09:39.906765 4755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 13:09:39.911635 4755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 13:09:39.911957 4755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 13:09:39.932919 4755 rpc_server.cc:307] RPC server started. Bound to: 127.1.124.188:39145
I20260808 13:09:39.932938 4860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.1.124.188:39145 every 8 connection(s)
I20260808 13:09:39.934448 4755 server_base.cc:1166] Dumped server information to /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-2/data/info.pb
I20260808 13:09:39.938299 4862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 13:09:39.941845 1522 external_mini_cluster.cc:1388] Started /tmp/dist-test-task9J1ueI/build/asan/bin/kudu as pid 4755
I20260808 13:09:39.942050 1522 external_mini_cluster.cc:1402] Reading /tmp/dist-test-task9J1ueI/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786194185613124-1522-0/minicluster-data/master-2/wal/instance
I20260808 13:09:39.942605 1522 external_mini_cluster.cc:1428] Setting key b42efaddc44674ec24ff27debbd7b96d
I20260808 13:09:39.945969 4641 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } attempt: 4
I20260808 13:09:39.946300 4862 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39499 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 13:09:39 dist-test-slave-t3q3 krb5kdc[4518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194578, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.190@KRBTEST.COM for kudu/127.1.124.188@KRBTEST.COM
Aug 08 13:09:39 dist-test-slave-t3q3 krb5kdc[4518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.188@KRBTEST.COM
Aug 08 13:09:39 dist-test-slave-t3q3 krb5kdc[4518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194579, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.188@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:09:39.967643 4645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.946639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:39145 (local address 127.0.0.1:50824)
0808 13:09:39.946849 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.946870 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:39.946996 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:39.948871 (+ 1875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:39.948877 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:39.949585 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:39.949819 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:39.949836 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.953203 (+ 3367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.953209 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:39.954247 (+ 1038us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:39.954259 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.955719 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.955725 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:39.955927 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:39.956649 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:39.956672 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:39.959356 (+ 2684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:39.963790 (+ 4434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:39.963796 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:39.963800 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:39.964435 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:39.964909 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:39.964914 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:39.964921 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:39.965114 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:39.966311 (+ 1197us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:39.966316 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:39.966529 (+ 213us) client_negotiation.cc:769] Sending connection context
0808 13:09:39.966812 (+ 283us) client_negotiation.cc:240] Negotiation successful
0808 13:09:39.967142 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 13:09:39.969113 4866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.946762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50824 (local address 127.1.124.188:39145)
0808 13:09:39.947465 (+ 703us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:39.947471 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:39.947502 (+ 31us) server_negotiation.cc:407] Connection header received
0808 13:09:39.947823 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:39.947856 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:39.948433 (+ 577us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:39.949540 (+ 1107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:39.950065 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.952984 (+ 2919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.954443 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.955505 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.956283 (+ 778us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:39.959622 (+ 3339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:39.959630 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:39.959642 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:39.959672 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:39.963648 (+ 3976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:39.964594 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:39.964601 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:39.964608 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:39.964772 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:39.965254 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:39.965257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:39.965260 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:39.966073 (+ 813us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:39.966350 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:39.967114 (+ 764us) server_negotiation.cc:299] Negotiation successful
0808 13:09:39.967692 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":81,"server-negotiator.queue_time_us":532,"spinlock_wait_cycles":49920,"thread_start_us":373,"threads_started":1}
I20260808 13:09:39.969292 4865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.944534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50822 (local address 127.1.124.188:39145)
0808 13:09:39.945426 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:39.945438 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:39.945482 (+ 44us) server_negotiation.cc:407] Connection header received
0808 13:09:39.947823 (+ 2341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:39.947856 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:39.948433 (+ 577us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:39.948877 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:39.950718 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.952999 (+ 2281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.954854 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.955495 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.955727 (+ 232us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:39.960719 (+ 4992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:39.960725 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:39.960730 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:39.960770 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:39.964805 (+ 4035us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:39.965875 (+ 1070us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:39.965883 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:39.965887 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:39.966079 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:39.966626 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:39.966630 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:39.966633 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:39.966983 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:39.967267 (+ 284us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:39.968653 (+ 1386us) server_negotiation.cc:299] Negotiation successful
0808 13:09:39.968876 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":550,"server-negotiator.queue_time_us":675,"thread_start_us":462,"threads_started":1}
I20260808 13:09:39.970732 4868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.951102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57884 (local address 127.1.124.190:39499)
0808 13:09:39.951566 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:39.951572 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:39.952625 (+ 1053us) server_negotiation.cc:407] Connection header received
0808 13:09:39.952829 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:39.952838 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:39.952920 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:39.953066 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:39.954466 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.955505 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.957590 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.958396 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.958544 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:39.963271 (+ 4727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:39.963280 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:39.963285 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:39.963325 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:39.966961 (+ 3636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:39.968157 (+ 1196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:39.968163 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:39.968166 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:39.968309 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:39.968844 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:39.968849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:39.968852 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:39.969265 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:39.969411 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:39.970111 (+ 700us) server_negotiation.cc:299] Negotiation successful
0808 13:09:39.970320 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":193,"threads_started":1}
I20260808 13:09:39.970785 4869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.951663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.190:39499 (local address 127.0.0.1:57884)
0808 13:09:39.952475 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.952525 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:39.952681 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:39.953145 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:39.953155 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:39.953998 (+ 843us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:39.954311 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:39.954323 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.955723 (+ 1400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.955728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:39.957394 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:39.957410 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.958656 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.958667 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:39.958865 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:39.959612 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:39.959656 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:39.963029 (+ 3373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:39.967115 (+ 4086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:39.967129 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:39.967136 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:39.967986 (+ 850us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:39.968509 (+ 523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:39.968513 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:39.968517 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:39.968704 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:39.969514 (+ 810us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:39.969521 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:39.969694 (+ 173us) client_negotiation.cc:769] Sending connection context
0808 13:09:39.969977 (+ 283us) client_negotiation.cc:240] Negotiation successful
0808 13:09:39.970305 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"spinlock_wait_cycles":896,"thread_start_us":380,"threads_started":1}
I20260808 13:09:39.972340 4862 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:09:39.977697 4641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99: Bootstrap starting.
Aug 08 13:09:39 dist-test-slave-t3q3 krb5kdc[4518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194579, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.188@KRBTEST.COM for kudu/127.1.124.189@KRBTEST.COM
I20260808 13:09:39.981838 4641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:39.983352 4641 log.cc:824] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99: Log is configured to *not* fsync() on all Append() calls
Aug 08 13:09:39 dist-test-slave-t3q3 krb5kdc[4518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.190@KRBTEST.COM
I20260808 13:09:39.988595 4869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.973136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.189:45787 (local address 127.0.0.1:39368)
0808 13:09:39.973320 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.973346 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:39.973426 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:39.973740 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:39.973743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:39.974093 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:39.974227 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:39.974235 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.975629 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.975634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:39.976702 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:39.976715 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.977701 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.977705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:39.977919 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:39.978610 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:39.978637 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:39.981464 (+ 2827us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:39.985463 (+ 3999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:39.985470 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:39.985485 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:39.986085 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:39.986616 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:39.986621 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:39.986625 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:39.986846 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:39.987545 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:39.987553 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:39.987734 (+ 181us) client_negotiation.cc:769] Sending connection context
0808 13:09:39.987915 (+ 181us) client_negotiation.cc:240] Negotiation successful
0808 13:09:39.988236 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 13:09:39.988894 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.973211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39368 (local address 127.1.124.189:45787)
0808 13:09:39.973405 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:39.973411 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:39.973431 (+ 20us) server_negotiation.cc:407] Connection header received
0808 13:09:39.973523 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:39.973528 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:39.973609 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:39.973728 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:39.974389 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.975423 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.976859 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.977520 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.977671 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:39.981719 (+ 4048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:39.981729 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:39.981738 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:39.981792 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:39.985309 (+ 3517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:39.986268 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:39.986276 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:39.986279 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:39.986467 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:39.986982 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:39.986987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:39.986990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:39.987372 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:39.987547 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:39.988035 (+ 488us) server_negotiation.cc:299] Negotiation successful
0808 13:09:39.988330 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 13:09:39.990002 4862 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } has no permanent_uuid. Determining permanent_uuid...
I20260808 13:09:39.990324 4641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99: No bootstrap required, opened a new log
I20260808 13:09:39.995944 4868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.978322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57900 (local address 127.1.124.190:39499)
0808 13:09:39.978495 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:39.978500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:39.978721 (+ 221us) server_negotiation.cc:407] Connection header received
0808 13:09:39.978912 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:39.978916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:39.979012 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:39.979119 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:39.980738 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.981519 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.983059 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.983540 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.983692 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:39.988911 (+ 5219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:39.988920 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:39.988925 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:39.988965 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:39.992908 (+ 3943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:39.993769 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:39.993777 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:39.993784 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:39.993944 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:39.994347 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:39.994352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:39.994355 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:39.994675 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:39.994996 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:39.995347 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 13:09:39.995599 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 13:09:39 dist-test-slave-t3q3 krb5kdc[4518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194579, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.188@KRBTEST.COM for kudu/127.1.124.188@KRBTEST.COM
I20260808 13:09:40.002051 4641 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11b6f813a9d84142b8ac90ec86bdff99" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39499 } } peers { permanent_uuid: "1f18f53a26a7435fb2b7dffae1cede93" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } } peers { permanent_uuid: "9c749554f46c40898177dcb0dc176ffa" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } }
I20260808 13:09:40.002507 4641 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11b6f813a9d84142b8ac90ec86bdff99, State: Initialized, Role: FOLLOWER
I20260808 13:09:40.003093 4641 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11b6f813a9d84142b8ac90ec86bdff99" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39499 } } peers { permanent_uuid: "1f18f53a26a7435fb2b7dffae1cede93" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } } peers { permanent_uuid: "9c749554f46c40898177dcb0dc176ffa" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } }
Aug 08 13:09:40 dist-test-slave-t3q3 krb5kdc[4518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194578, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.1.124.189@KRBTEST.COM
I20260808 13:09:40.006461 4872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11b6f813a9d84142b8ac90ec86bdff99" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39499 } } peers { permanent_uuid: "1f18f53a26a7435fb2b7dffae1cede93" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } } peers { permanent_uuid: "9c749554f46c40898177dcb0dc176ffa" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } } }
I20260808 13:09:40.007872 4641 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:09:40.008025 4872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:09:40.009406 4869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.990993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:39145 (local address 127.0.0.1:50832)
0808 13:09:39.991169 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:39.991190 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:39.991296 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:39.991980 (+ 684us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:39.991986 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:39.992516 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:39.992724 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:39.992737 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.993823 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.993829 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:39.995192 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:39.995207 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.996366 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.996373 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:39.996571 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:39.997236 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:39.997259 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:39.999989 (+ 2730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:40.005347 (+ 5358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:40.005353 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:40.005357 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:40.006671 (+ 1314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:40.007109 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:40.007114 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:40.007118 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:40.007287 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:40.008268 (+ 981us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:40.008274 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:40.008434 (+ 160us) client_negotiation.cc:769] Sending connection context
0808 13:09:40.008647 (+ 213us) client_negotiation.cc:240] Negotiation successful
0808 13:09:40.008969 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260808 13:09:40.010756 4866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.991334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50832 (local address 127.1.124.188:39145)
0808 13:09:39.991547 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:39.991553 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:39.991570 (+ 17us) server_negotiation.cc:407] Connection header received
0808 13:09:39.991666 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:39.991671 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:39.991814 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:39.991924 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:39.992929 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.993681 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.995352 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:39.996007 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:39.996209 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:40.000220 (+ 4011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:40.000226 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:40.000230 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:40.000266 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:40.005173 (+ 4907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:40.006805 (+ 1632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:40.006810 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:40.006813 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:40.006972 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:40.007422 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:40.007427 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:40.007430 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:40.007850 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:40.008246 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:40.009998 (+ 1752us) server_negotiation.cc:299] Negotiation successful
0808 13:09:40.010296 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 13:09:40.015669 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:39.997814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39376 (local address 127.1.124.189:45787)
0808 13:09:39.997977 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:39.997984 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:39.998031 (+ 47us) server_negotiation.cc:407] Connection header received
0808 13:09:39.998229 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:39.998234 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:39.998306 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:39.998412 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:39.999958 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.000748 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.002221 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.002900 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.003035 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:40.007718 (+ 4683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:40.007726 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:40.007732 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:40.007778 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:40.011608 (+ 3830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:40.012519 (+ 911us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:40.012526 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:40.012531 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:40.012705 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:40.013239 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:40.013245 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:40.013248 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:40.013582 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:40.014954 (+ 1372us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:40.015089 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 13:09:40.015338 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 13:09:40.020586 4887 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:09:40.020759 4887 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:09:40.021040 1522 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 13:09:40.024224 4604 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "11b6f813a9d84142b8ac90ec86bdff99"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57900
I20260808 13:09:40.024700 4604 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 13:09:40.024917 4604 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:40.026008 4862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c749554f46c40898177dcb0dc176ffa: Bootstrap starting.
I20260808 13:09:40.028926 4604 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11b6f813a9d84142b8ac90ec86bdff99" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39499 } } peers { permanent_uuid: "1f18f53a26a7435fb2b7dffae1cede93" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } } peers { permanent_uuid: "9c749554f46c40898177dcb0dc176ffa" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } }
I20260808 13:09:40.020699 4534 master_runner.cc:186] Error getting master registration for 127.1.124.189:45787: OK, instance_id { permanent_uuid: "1f18f53a26a7435fb2b7dffae1cede93" instance_seqno: 1786194579341951 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 13:09:40.030529 4862 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c749554f46c40898177dcb0dc176ffa: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:40.031498 4604 leader_election.cc:288] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99 [CANDIDATE]: Term 1 election: Requested vote from peers 1f18f53a26a7435fb2b7dffae1cede93 (127.1.124.189:45787), 9c749554f46c40898177dcb0dc176ffa (127.1.124.188:39145)
I20260808 13:09:40.031630 4534 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:09:40.032240 4862 log.cc:824] T 00000000000000000000000000000000 P 9c749554f46c40898177dcb0dc176ffa: Log is configured to *not* fsync() on all Append() calls
I20260808 13:09:40.036304 4862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c749554f46c40898177dcb0dc176ffa: No bootstrap required, opened a new log
W20260808 13:09:40.041391 4567 tablet.cc:2369] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:09:40.050107 4862 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c749554f46c40898177dcb0dc176ffa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11b6f813a9d84142b8ac90ec86bdff99" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39499 } } peers { permanent_uuid: "1f18f53a26a7435fb2b7dffae1cede93" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } } peers { permanent_uuid: "9c749554f46c40898177dcb0dc176ffa" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } }
I20260808 13:09:40.050729 4862 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c749554f46c40898177dcb0dc176ffa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c749554f46c40898177dcb0dc176ffa, State: Initialized, Role: FOLLOWER
I20260808 13:09:40.051383 4862 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c749554f46c40898177dcb0dc176ffa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11b6f813a9d84142b8ac90ec86bdff99" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39499 } } peers { permanent_uuid: "1f18f53a26a7435fb2b7dffae1cede93" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } } peers { permanent_uuid: "9c749554f46c40898177dcb0dc176ffa" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } }
I20260808 13:09:40.058532 4898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c749554f46c40898177dcb0dc176ffa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11b6f813a9d84142b8ac90ec86bdff99" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39499 } } peers { permanent_uuid: "1f18f53a26a7435fb2b7dffae1cede93" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } } peers { permanent_uuid: "9c749554f46c40898177dcb0dc176ffa" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } } }
I20260808 13:09:40.060928 4898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c749554f46c40898177dcb0dc176ffa [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:09:40.060774 4862 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c749554f46c40898177dcb0dc176ffa [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:09:40.060894 4820 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11b6f813a9d84142b8ac90ec86bdff99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "9c749554f46c40898177dcb0dc176ffa"
I20260808 13:09:40.061474 4820 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c749554f46c40898177dcb0dc176ffa [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:40.065274 4820 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9c749554f46c40898177dcb0dc176ffa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11b6f813a9d84142b8ac90ec86bdff99 in term 1.
I20260808 13:09:40.066164 4563 leader_election.cc:302] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11b6f813a9d84142b8ac90ec86bdff99, 9c749554f46c40898177dcb0dc176ffa; no voters:
I20260808 13:09:40.066787 4872 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 13:09:40.067497 4872 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99 [term 1 LEADER]: Becoming Leader. State: Replica: 11b6f813a9d84142b8ac90ec86bdff99, State: Running, Role: LEADER
I20260808 13:09:40.068140 4872 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11b6f813a9d84142b8ac90ec86bdff99" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39499 } } peers { permanent_uuid: "1f18f53a26a7435fb2b7dffae1cede93" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } } peers { permanent_uuid: "9c749554f46c40898177dcb0dc176ffa" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } }
I20260808 13:09:40.088501 4868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:40.038814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57904 (local address 127.1.124.190:39499)
0808 13:09:40.038994 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.039002 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.054288 (+ 15286us) server_negotiation.cc:407] Connection header received
0808 13:09:40.056849 (+ 2561us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.056859 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.056969 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.057072 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:40.069251 (+ 12179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.070376 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.073126 (+ 2750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.073820 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.073955 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:40.079117 (+ 5162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:40.079124 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:40.079129 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:40.079170 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:40.084871 (+ 5701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:40.085776 (+ 905us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:40.085781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:40.085784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:40.085940 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:40.086399 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:40.086404 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:40.086407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:40.086794 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:40.087072 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:40.087786 (+ 714us) server_negotiation.cc:299] Negotiation successful
0808 13:09:40.088100 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 13:09:40.089190 4907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11b6f813a9d84142b8ac90ec86bdff99. Latest consensus state: current_term: 1 leader_uuid: "11b6f813a9d84142b8ac90ec86bdff99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11b6f813a9d84142b8ac90ec86bdff99" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39499 } } peers { permanent_uuid: "1f18f53a26a7435fb2b7dffae1cede93" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } } peers { permanent_uuid: "9c749554f46c40898177dcb0dc176ffa" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } } }
I20260808 13:09:40.089709 4907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:40.090823 4866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:40.045045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50848 (local address 127.1.124.188:39145)
0808 13:09:40.045235 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.045242 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.054950 (+ 9708us) server_negotiation.cc:407] Connection header received
0808 13:09:40.055822 (+ 872us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.055831 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.055934 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.056035 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:40.059649 (+ 3614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.060745 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.068701 (+ 7956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.069698 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.069861 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:40.074893 (+ 5032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:40.074903 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:40.074911 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:40.074965 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:40.081375 (+ 6410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:40.083472 (+ 2097us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:40.083479 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:40.083484 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:40.083875 (+ 391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:40.084852 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:40.084860 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:40.084867 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:40.085813 (+ 946us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:40.088547 (+ 2734us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:40.089251 (+ 704us) server_negotiation.cc:299] Negotiation successful
0808 13:09:40.089849 (+ 598us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 13:09:40.091528 4743 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } attempt: 5
I20260808 13:09:40.104914 4914 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 13:09:40.107021 4914 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 08 13:09:40 dist-test-slave-t3q3 krb5kdc[4518](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786194579, etypes {rep=17 tkt=17 ses=17}, kudu/127.1.124.189@KRBTEST.COM for kudu/127.1.124.188@KRBTEST.COM
I20260808 13:09:40.114719 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:40.039100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39382 (local address 127.1.124.189:45787)
0808 13:09:40.039278 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.039284 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.052115 (+ 12831us) server_negotiation.cc:407] Connection header received
0808 13:09:40.052679 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.052686 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.052775 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.052855 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:40.081570 (+ 28715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.082894 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.095520 (+ 12626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.096332 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.096443 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:40.102296 (+ 5853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:40.102303 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:40.102312 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:40.102357 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:40.106798 (+ 4441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:40.109261 (+ 2463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:40.109271 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:40.109277 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:40.109465 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:40.112056 (+ 2591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:40.112077 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:40.112080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:40.112455 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:40.112726 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:40.114120 (+ 1394us) server_negotiation.cc:299] Negotiation successful
0808 13:09:40.114363 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 13:09:40.121057 4750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:40.092761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.1.124.188:39145 (local address 127.0.0.1:50850)
0808 13:09:40.092996 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 13:09:40.093021 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 13:09:40.093167 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 13:09:40.095364 (+ 2197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 13:09:40.095371 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 13:09:40.096106 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0808 13:09:40.096412 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:40.096429 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.098647 (+ 2218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.098656 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:40.100464 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 13:09:40.100485 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.101917 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.101926 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 13:09:40.102330 (+ 404us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:40.107211 (+ 4881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 13:09:40.107245 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 13:09:40.110877 (+ 3632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 13:09:40.117457 (+ 6580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:40.117465 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:40.117468 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:40.118021 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:40.118561 (+ 540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 13:09:40.118568 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 13:09:40.118572 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 13:09:40.118739 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 13:09:40.119663 (+ 924us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 13:09:40.119669 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 13:09:40.119877 (+ 208us) client_negotiation.cc:769] Sending connection context
0808 13:09:40.120052 (+ 175us) client_negotiation.cc:240] Negotiation successful
0808 13:09:40.120507 (+ 455us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 13:09:40.122217 4866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:40.093174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50850 (local address 127.1.124.188:39145)
0808 13:09:40.095043 (+ 1869us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.095050 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.095071 (+ 21us) server_negotiation.cc:407] Connection header received
0808 13:09:40.095128 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.095132 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.095207 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.095313 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:40.096686 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.098448 (+ 1762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.100717 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.101696 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.109674 (+ 7978us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:40.111166 (+ 1492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 13:09:40.111173 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 13:09:40.111177 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 13:09:40.111211 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 13:09:40.117312 (+ 6101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:40.118188 (+ 876us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:40.118194 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:40.118197 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:40.118378 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 13:09:40.118883 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 13:09:40.118887 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 13:09:40.118890 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 13:09:40.119366 (+ 476us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 13:09:40.119577 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:40.121379 (+ 1802us) server_negotiation.cc:299] Negotiation successful
0808 13:09:40.121699 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1722}
I20260808 13:09:40.130201 4743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f18f53a26a7435fb2b7dffae1cede93: Bootstrap starting.
W20260808 13:09:40.132820 4917 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c749554f46c40898177dcb0dc176ffa: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 13:09:40.133131 4917 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 13:09:40.136770 4820 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9c749554f46c40898177dcb0dc176ffa [term 1 FOLLOWER]: Refusing update from remote peer 11b6f813a9d84142b8ac90ec86bdff99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:09:40.137646 4907 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c749554f46c40898177dcb0dc176ffa" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:09:40.117048 4755 master_runner.cc:186] Error getting master registration for 127.1.124.189:45787: OK, instance_id { permanent_uuid: "1f18f53a26a7435fb2b7dffae1cede93" instance_seqno: 1786194579341951 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 13:09:40.143579 4755 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 13:09:40.148154 4743 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f18f53a26a7435fb2b7dffae1cede93: Neither blocks nor log segments found. Creating new log.
I20260808 13:09:40.149564 4743 log.cc:824] T 00000000000000000000000000000000 P 1f18f53a26a7435fb2b7dffae1cede93: Log is configured to *not* fsync() on all Append() calls
I20260808 13:09:40.159130 4743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f18f53a26a7435fb2b7dffae1cede93: No bootstrap required, opened a new log
I20260808 13:09:40.162874 4898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c749554f46c40898177dcb0dc176ffa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11b6f813a9d84142b8ac90ec86bdff99. Latest consensus state: current_term: 1 leader_uuid: "11b6f813a9d84142b8ac90ec86bdff99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11b6f813a9d84142b8ac90ec86bdff99" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39499 } } peers { permanent_uuid: "1f18f53a26a7435fb2b7dffae1cede93" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } } peers { permanent_uuid: "9c749554f46c40898177dcb0dc176ffa" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } } }
I20260808 13:09:40.163200 4898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c749554f46c40898177dcb0dc176ffa [sys.catalog]: This master's current role is: FOLLOWER
W20260808 13:09:40.164213 4787 tablet.cc:2369] T 00000000000000000000000000000000 P 9c749554f46c40898177dcb0dc176ffa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 13:09:40.167846 4920 mvcc.cc:205] Tried to move back new op lower bound from 7316253000202223616 to 7316252999964647424. Current Snapshot: MvccSnapshot[applied={T|T < 7316253000202223616}]
I20260808 13:09:40.169734 4921 mvcc.cc:205] Tried to move back new op lower bound from 7316253000202223616 to 7316252999964647424. Current Snapshot: MvccSnapshot[applied={T|T < 7316253000202223616}]
I20260808 13:09:40.170653 4743 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f18f53a26a7435fb2b7dffae1cede93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11b6f813a9d84142b8ac90ec86bdff99" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39499 } } peers { permanent_uuid: "1f18f53a26a7435fb2b7dffae1cede93" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } } peers { permanent_uuid: "9c749554f46c40898177dcb0dc176ffa" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } }
I20260808 13:09:40.171094 4743 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f18f53a26a7435fb2b7dffae1cede93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f18f53a26a7435fb2b7dffae1cede93, State: Initialized, Role: FOLLOWER
I20260808 13:09:40.171618 4743 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f18f53a26a7435fb2b7dffae1cede93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11b6f813a9d84142b8ac90ec86bdff99" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39499 } } peers { permanent_uuid: "1f18f53a26a7435fb2b7dffae1cede93" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } } peers { permanent_uuid: "9c749554f46c40898177dcb0dc176ffa" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } }
I20260808 13:09:40.183435 4926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f18f53a26a7435fb2b7dffae1cede93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11b6f813a9d84142b8ac90ec86bdff99" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39499 } } peers { permanent_uuid: "1f18f53a26a7435fb2b7dffae1cede93" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } } peers { permanent_uuid: "9c749554f46c40898177dcb0dc176ffa" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } } }
I20260808 13:09:40.185492 4926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f18f53a26a7435fb2b7dffae1cede93 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:09:40.185303 4743 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f18f53a26a7435fb2b7dffae1cede93 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 13:09:40.185745 4707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f18f53a26a7435fb2b7dffae1cede93 [term 0 FOLLOWER]: Advancing to term 1
I20260808 13:09:40.188305 4709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "11b6f813a9d84142b8ac90ec86bdff99" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1f18f53a26a7435fb2b7dffae1cede93"
I20260808 13:09:40.190593 4707 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1f18f53a26a7435fb2b7dffae1cede93 [term 1 FOLLOWER]: Refusing update from remote peer 11b6f813a9d84142b8ac90ec86bdff99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 13:09:40.194325 4709 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1f18f53a26a7435fb2b7dffae1cede93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11b6f813a9d84142b8ac90ec86bdff99 in term 1.
I20260808 13:09:40.196919 4872 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f18f53a26a7435fb2b7dffae1cede93" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 13:09:40.211580 4872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11b6f813a9d84142b8ac90ec86bdff99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11b6f813a9d84142b8ac90ec86bdff99" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39499 } } peers { permanent_uuid: "1f18f53a26a7435fb2b7dffae1cede93" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } } peers { permanent_uuid: "9c749554f46c40898177dcb0dc176ffa" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } } }
I20260808 13:09:40.211975 4872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:40.213420 4898 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c749554f46c40898177dcb0dc176ffa [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "11b6f813a9d84142b8ac90ec86bdff99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11b6f813a9d84142b8ac90ec86bdff99" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39499 } } peers { permanent_uuid: "1f18f53a26a7435fb2b7dffae1cede93" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } } peers { permanent_uuid: "9c749554f46c40898177dcb0dc176ffa" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } } }
I20260808 13:09:40.213784 4898 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c749554f46c40898177dcb0dc176ffa [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:09:40.217316 4914 catalog_manager.cc:1347] Generated new cluster ID: af1565ada1e34936aa41b5e5f4ecad6d
I20260808 13:09:40.217433 4914 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 13:09:40.251144 4926 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f18f53a26a7435fb2b7dffae1cede93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11b6f813a9d84142b8ac90ec86bdff99. Latest consensus state: current_term: 1 leader_uuid: "11b6f813a9d84142b8ac90ec86bdff99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11b6f813a9d84142b8ac90ec86bdff99" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39499 } } peers { permanent_uuid: "1f18f53a26a7435fb2b7dffae1cede93" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } } peers { permanent_uuid: "9c749554f46c40898177dcb0dc176ffa" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } } }
I20260808 13:09:40.251525 4926 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f18f53a26a7435fb2b7dffae1cede93 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:09:40.253793 4946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f18f53a26a7435fb2b7dffae1cede93 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "11b6f813a9d84142b8ac90ec86bdff99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11b6f813a9d84142b8ac90ec86bdff99" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39499 } } peers { permanent_uuid: "1f18f53a26a7435fb2b7dffae1cede93" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } } peers { permanent_uuid: "9c749554f46c40898177dcb0dc176ffa" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } } }
I20260808 13:09:40.254921 4946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f18f53a26a7435fb2b7dffae1cede93 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 13:09:40.261011 4907 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "11b6f813a9d84142b8ac90ec86bdff99" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11b6f813a9d84142b8ac90ec86bdff99" member_type: VOTER last_known_addr { host: "127.1.124.190" port: 39499 } } peers { permanent_uuid: "1f18f53a26a7435fb2b7dffae1cede93" member_type: VOTER last_known_addr { host: "127.1.124.189" port: 45787 } } peers { permanent_uuid: "9c749554f46c40898177dcb0dc176ffa" member_type: VOTER last_known_addr { host: "127.1.124.188" port: 39145 } } }
I20260808 13:09:40.261299 4907 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99 [sys.catalog]: This master's current role is: LEADER
I20260808 13:09:40.266028 4914 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 13:09:40.268303 4914 catalog_manager.cc:1504] Loading token signing keys...
I20260808 13:09:40.283545 4914 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11b6f813a9d84142b8ac90ec86bdff99: Generated new TSK 0
I20260808 13:09:40.284670 4914 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 13:09:40.291569 4948 catalog_manager.cc:1259] Loaded cluster ID: af1565ada1e34936aa41b5e5f4ecad6d
I20260808 13:09:40.291766 4948 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1f18f53a26a7435fb2b7dffae1cede93: loading cluster ID for follower catalog manager: success
I20260808 13:09:40.295509 4948 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1f18f53a26a7435fb2b7dffae1cede93: acquiring CA information for follower catalog manager: success
I20260808 13:09:40.298120 4948 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1f18f53a26a7435fb2b7dffae1cede93: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 13:09:40.428485 1522 mini_kdc.cc:348] Time spent kdestroy: real 0.118s user 0.001s sys 0.030s
W20260808 13:09:40.483717 4868 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:40.465546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57906 (local address 127.1.124.190:39499)
0808 13:09:40.465796 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.465805 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.480768 (+ 14963us) server_negotiation.cc:407] Connection header received
0808 13:09:40.481163 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.481170 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.481276 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.481388 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:40.483330 (+ 1942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57906: BlockingRecv error: recv got EOF from 127.0.0.1:57906 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 13:09:40.486936 4752 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:40.472582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39396 (local address 127.1.124.189:45787)
0808 13:09:40.472847 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.472854 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.484599 (+ 11745us) server_negotiation.cc:407] Connection header received
0808 13:09:40.484922 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.484933 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.485092 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.485197 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:40.486617 (+ 1420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39396: BlockingRecv error: recv got EOF from 127.0.0.1:39396 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
W20260808 13:09:40.489538 4866 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:40.481236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50864 (local address 127.1.124.188:39145)
0808 13:09:40.481676 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.481684 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.486111 (+ 4427us) server_negotiation.cc:407] Connection header received
0808 13:09:40.487295 (+ 1184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.487302 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.487403 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.487492 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:40.489218 (+ 1726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50864: BlockingRecv error: recv got EOF from 127.0.0.1:50864 (error 108)
Metrics: {"server-negotiator.queue_time_us":291}
I20260808 13:09:40.541493 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:40.523198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39404 (local address 127.1.124.189:45787)
0808 13:09:40.523418 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.523425 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.532758 (+ 9333us) server_negotiation.cc:407] Connection header received
0808 13:09:40.533088 (+ 330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.533095 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.533328 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.533537 (+ 209us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:09:40.534330 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.535639 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.538022 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.538830 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.538938 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:40.539792 (+ 854us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:09:40.540589 (+ 797us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:09:40.540729 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:40.541008 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 13:09:40.541123 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260808 13:09:40.579116 4866 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:40.569649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50874 (local address 127.1.124.188:39145)
0808 13:09:40.569883 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.569891 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.576833 (+ 6942us) server_negotiation.cc:407] Connection header received
0808 13:09:40.577423 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.577430 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.577612 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.577705 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:40.578876 (+ 1171us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50874: BlockingRecv error: recv got EOF from 127.0.0.1:50874 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 13:09:40.627774 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:40.613423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39406 (local address 127.1.124.189:45787)
0808 13:09:40.613663 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.613669 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.618817 (+ 5148us) server_negotiation.cc:407] Connection header received
0808 13:09:40.620477 (+ 1660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.620485 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.620624 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.620707 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:09:40.621522 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.622831 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.625174 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.625891 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.625996 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:40.626747 (+ 751us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:09:40.626928 (+ 181us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:09:40.627000 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:40.627287 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 13:09:40.627396 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260808 13:09:40.658738 4866 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:40.654709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50888 (local address 127.1.124.188:39145)
0808 13:09:40.654936 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.654944 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.656695 (+ 1751us) server_negotiation.cc:407] Connection header received
0808 13:09:40.657048 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.657056 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.657160 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.657246 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:40.658423 (+ 1177us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50888: BlockingRecv error: recv got EOF from 127.0.0.1:50888 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 13:09:40.673964 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:40.666501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39408 (local address 127.1.124.189:45787)
0808 13:09:40.666728 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.666735 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.666758 (+ 23us) server_negotiation.cc:407] Connection header received
0808 13:09:40.666953 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.666962 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.667080 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.667152 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:09:40.667900 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.669371 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.671442 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.672244 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.672405 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:40.673004 (+ 599us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:09:40.673183 (+ 179us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:09:40.673293 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:40.673482 (+ 189us) server_negotiation.cc:299] Negotiation successful
0808 13:09:40.673590 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260808 13:09:40.686830 4866 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:40.681671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50892 (local address 127.1.124.188:39145)
0808 13:09:40.681888 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.681895 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.684763 (+ 2868us) server_negotiation.cc:407] Connection header received
0808 13:09:40.685152 (+ 389us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.685159 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.685267 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.685359 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:40.686548 (+ 1189us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50892: BlockingRecv error: recv got EOF from 127.0.0.1:50892 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260808 13:09:40.703785 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:40.695554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39412 (local address 127.1.124.189:45787)
0808 13:09:40.695779 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.695786 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.696536 (+ 750us) server_negotiation.cc:407] Connection header received
0808 13:09:40.696911 (+ 375us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.696917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.697019 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.697113 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:09:40.697835 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.699076 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.701302 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.702058 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.702168 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:40.702890 (+ 722us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:09:40.703037 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:09:40.703119 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:40.703375 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 13:09:40.703498 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
W20260808 13:09:40.714442 4866 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:40.711539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50902 (local address 127.1.124.188:39145)
0808 13:09:40.711836 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.711844 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.712440 (+ 596us) server_negotiation.cc:407] Connection header received
0808 13:09:40.712777 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.712784 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.712915 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.713021 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:40.714169 (+ 1148us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50902: BlockingRecv error: recv got EOF from 127.0.0.1:50902 (error 108)
Metrics: {"server-negotiator.queue_time_us":172}
I20260808 13:09:40.746277 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:40.738232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39424 (local address 127.1.124.189:45787)
0808 13:09:40.738445 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.738451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.739027 (+ 576us) server_negotiation.cc:407] Connection header received
0808 13:09:40.739383 (+ 356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.739390 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.739491 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.739564 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:09:40.740287 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.741566 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.743695 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.744452 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.744577 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:40.745284 (+ 707us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:09:40.745464 (+ 180us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:09:40.745558 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:40.745818 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 13:09:40.745920 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260808 13:09:40.757082 4866 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:40.754091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50910 (local address 127.1.124.188:39145)
0808 13:09:40.754283 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.754290 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.755220 (+ 930us) server_negotiation.cc:407] Connection header received
0808 13:09:40.755461 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.755468 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.755606 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.755695 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:40.756832 (+ 1137us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50910: BlockingRecv error: recv got EOF from 127.0.0.1:50910 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 13:09:40.787981 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:40.780488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39440 (local address 127.1.124.189:45787)
0808 13:09:40.780688 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.780694 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.781252 (+ 558us) server_negotiation.cc:407] Connection header received
0808 13:09:40.781591 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.781599 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.781701 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.781839 (+ 138us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:09:40.782335 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.783575 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.785669 (+ 2094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.786357 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.786489 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:40.787137 (+ 648us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:09:40.787279 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:09:40.787342 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:40.787606 (+ 264us) server_negotiation.cc:299] Negotiation successful
0808 13:09:40.787705 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260808 13:09:40.802327 4866 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:40.799971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50920 (local address 127.1.124.188:39145)
0808 13:09:40.800204 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.800211 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.800758 (+ 547us) server_negotiation.cc:407] Connection header received
0808 13:09:40.800943 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.800950 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.801059 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.801182 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:40.802090 (+ 908us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50920: BlockingRecv error: recv got EOF from 127.0.0.1:50920 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 13:09:40.843256 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:40.838031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39456 (local address 127.1.124.189:45787)
0808 13:09:40.838210 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.838215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.838640 (+ 425us) server_negotiation.cc:407] Connection header received
0808 13:09:40.838807 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.838811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.838917 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.839009 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:09:40.839429 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.840233 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.841626 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.842096 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.842205 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:40.842613 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:09:40.842725 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:09:40.842795 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:40.842938 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 13:09:40.843006 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 13:09:40.850816 4866 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:40.848861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50926 (local address 127.1.124.188:39145)
0808 13:09:40.849141 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.849147 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.849405 (+ 258us) server_negotiation.cc:407] Connection header received
0808 13:09:40.849648 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.849653 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.849724 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.849833 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:40.850634 (+ 801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50926: BlockingRecv error: recv got EOF from 127.0.0.1:50926 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260808 13:09:40.924391 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:40.918982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39462 (local address 127.1.124.189:45787)
0808 13:09:40.919272 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.919281 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.919624 (+ 343us) server_negotiation.cc:407] Connection header received
0808 13:09:40.919887 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.919891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.919960 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.920069 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:09:40.920523 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.921263 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.922620 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:40.923046 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:40.923196 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:40.923605 (+ 409us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:09:40.923729 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:09:40.923834 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:40.923987 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 13:09:40.924096 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
W20260808 13:09:40.932494 4866 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:40.930474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50932 (local address 127.1.124.188:39145)
0808 13:09:40.930746 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:40.930754 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:40.931039 (+ 285us) server_negotiation.cc:407] Connection header received
0808 13:09:40.931268 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:40.931273 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:40.931347 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:40.931469 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:40.932298 (+ 829us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50932: BlockingRecv error: recv got EOF from 127.0.0.1:50932 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
I20260808 13:09:41.070617 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:41.064849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39476 (local address 127.1.124.189:45787)
0808 13:09:41.065159 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:41.065167 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:41.065543 (+ 376us) server_negotiation.cc:407] Connection header received
0808 13:09:41.065837 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:41.065844 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:41.065928 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:41.066044 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:09:41.066501 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:41.067287 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:41.068815 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:41.069279 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:41.069427 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:41.069873 (+ 446us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:09:41.069985 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:09:41.070090 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:41.070254 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 13:09:41.070335 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
W20260808 13:09:41.078401 4866 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 13:09:41.076507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50948 (local address 127.1.124.188:39145)
0808 13:09:41.076678 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:41.076683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:41.077122 (+ 439us) server_negotiation.cc:407] Connection header received
0808 13:09:41.077272 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:41.077275 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:41.077360 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:41.077452 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 13:09:41.078161 (+ 709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50948: BlockingRecv error: recv got EOF from 127.0.0.1:50948 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 13:09:41.135406 4917 catalog_manager.cc:1259] Loaded cluster ID: af1565ada1e34936aa41b5e5f4ecad6d
I20260808 13:09:41.135512 4917 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9c749554f46c40898177dcb0dc176ffa: loading cluster ID for follower catalog manager: success
I20260808 13:09:41.138120 4917 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9c749554f46c40898177dcb0dc176ffa: acquiring CA information for follower catalog manager: success
I20260808 13:09:41.140100 4917 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9c749554f46c40898177dcb0dc176ffa: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 13:09:41.343699 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:41.338313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39492 (local address 127.1.124.189:45787)
0808 13:09:41.338568 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:41.338575 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:41.338756 (+ 181us) server_negotiation.cc:407] Connection header received
0808 13:09:41.339041 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:41.339046 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:41.339115 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:41.339224 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:09:41.339662 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:41.340596 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:41.341980 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:41.342387 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:41.342529 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:41.342961 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:09:41.343082 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:09:41.343186 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:41.343340 (+ 154us) server_negotiation.cc:299] Negotiation successful
0808 13:09:41.343423 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 13:09:41.355798 4866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:41.349930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50956 (local address 127.1.124.188:39145)
0808 13:09:41.350156 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:41.350162 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:41.350366 (+ 204us) server_negotiation.cc:407] Connection header received
0808 13:09:41.350599 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:41.350622 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:41.350800 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:41.350930 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:09:41.351413 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:41.352191 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:41.353573 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:41.353985 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:41.354133 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:41.354674 (+ 541us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:09:41.355193 (+ 519us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:09:41.355377 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:41.355431 (+ 54us) server_negotiation.cc:299] Negotiation successful
0808 13:09:41.355514 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 13:09:41.380487 4866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:41.362091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50964 (local address 127.1.124.188:39145)
0808 13:09:41.363802 (+ 1711us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:41.363811 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:41.374664 (+ 10853us) server_negotiation.cc:407] Connection header received
0808 13:09:41.374910 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:41.374916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:41.375015 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:41.375121 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:09:41.375974 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:41.377034 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:41.378690 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:41.379094 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:41.379236 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:41.379678 (+ 442us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:09:41.379785 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:09:41.379888 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:41.380093 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 13:09:41.380182 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1525}
I20260808 13:09:41.381652 4752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:41.362315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39494 (local address 127.1.124.189:45787)
0808 13:09:41.362501 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:41.362509 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:41.374125 (+ 11616us) server_negotiation.cc:407] Connection header received
0808 13:09:41.374698 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:41.374705 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:41.374802 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:41.374902 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:09:41.375523 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:41.376720 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:41.379301 (+ 2581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:41.379962 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:41.380099 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:41.380861 (+ 762us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:09:41.381003 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:09:41.381104 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:41.381263 (+ 159us) server_negotiation.cc:299] Negotiation successful
0808 13:09:41.381348 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 13:09:41.382462 5115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 13:09:41.362041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57912 (local address 127.1.124.190:39499)
0808 13:09:41.373222 (+ 11181us) server_negotiation.cc:206] Beginning negotiation
0808 13:09:41.373232 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 13:09:41.374534 (+ 1302us) server_negotiation.cc:407] Connection header received
0808 13:09:41.374790 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 13:09:41.374796 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 13:09:41.375055 (+ 259us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 13:09:41.375287 (+ 232us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 13:09:41.375965 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:41.376754 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:41.379305 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 13:09:41.379945 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 13:09:41.380093 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 13:09:41.381186 (+ 1093us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 13:09:41.381692 (+ 506us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 13:09:41.381799 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 13:09:41.382108 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 13:09:41.382193 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10996,"thread_start_us":11207,"threads_started":1}
I20260808 13:09:41.386849 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 4534
I20260808 13:09:41.388998 4633 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:09:41.501116 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 4534
I20260808 13:09:41.529850 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 4649
I20260808 13:09:41.531106 4736 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:09:41.638877 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 4649
I20260808 13:09:41.670146 1522 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-task9J1ueI/build/asan/bin/kudu pid 4755
I20260808 13:09:41.671360 4853 generic_service.cc:196] Checking for leaks (request via RPC)
I20260808 13:09:41.778626 1522 external_mini_cluster.cc:1595] Killing /tmp/dist-test-task9J1ueI/build/asan/bin/kudu with pid 4755
2026-08-08T13:09:41Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5180 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10507 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (396086 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST